[HUDI-4122] Fix NPE caused by adding kafka nodes

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11.1
• Component/s: deltastreamer
• Labels: pull-request-available
• Assignee: Xianghu Wang
• Reporter: Xianghu Wang
• Created: 19/May/22 07:16
• Updated: 16/Aug/22 16:48
• Resolved: 21/May/22 08:30

Description

Yesterday, when we add more nodes to kafka cluster, some of our hudi tasks failed with a NPE as below:

Caused by: java.lang.NullPointerException
at org.apache.hudi.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:241)
at org.apache.hudi.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:67)
at org.apache.hudi.utilities.sources.Source.fetchNext(Source.java:76)
at org.apache.hudi.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:63)
at org.apache.hudi.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:430)
at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:283)
at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:641)

Issue Links

• links to: GitHub Pull Request #5632