[HUDI-1006] deltastreamer use kafkaSource with offset reset strategy: latest can't consume data

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: deltastreamer
• Labels: pull-request-available
• Assignee: Tianye Li
• Reporter: liujinhui
• Created: 08/Jun/20 07:19
• Updated: 14/Jun/20 10:05
• Resolved: 14/Jun/20 10:05

Description

org.apache.hudi.utilities.sources.JsonKafkaSource#fetchNewData

if (totalNewMsgs <= 0)

{ return new InputBatch<>(Option.empty(), lastCheckpointStr.isPresent() ? lastCheckpointStr.get() : ""); }

I think it should not be empty here, it should be

if (totalNewMsgs <= 0)

{ return new InputBatch<>(Option.empty(), lastCheckpointStr.isPresent() ? lastCheckpointStr.get() : CheckpointUtils.offsetsToStr(offsetRanges)); }

Issue Links

• links to: GitHub Pull Request #1719