[GOBBLIN-1483] Handle null valued ConsumerRecords in Kafka Streaming Extractor

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.15.0
• Fix Version/s: 0.17.0
• Component/s: gobblin-kafka
• Assignee: Sudarshan Vasudevan
• Reporter: Sudarshan Vasudevan
• Created: 25/Jun/21 02:16
• Updated: 14/Jun/23 04:13

Description

Currently, null-valued ConsumerRecords returned by the Kafka consumer are passed down the construct chain and can result in irrecoverable failures. Null valued records should be filtered out early in the pipeline so as to avoid such errors.