[FLINK-32732] auto offset reset should be exposed to user

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 1.16.1
• Component/s: Connectors / Kafka
• Labels: pull-request-available stale-major
• Assignee: Unassigned
• Reporter: xiaogang zhou
• Created: 02/Aug/23 08:07
• Updated: 15/Apr/24 09:28
• Resolved: 15/Apr/24 09:28

Description

// code placeholder
maybeOverride(
        ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,
        startingOffsetsInitializer.getAutoOffsetResetStrategy().name().toLowerCase(),
        true); 

now flink override the auto.offset.reset with the scan.startup.mode config, and user's explicit config does not take effect. I think maybe we should expose this to customer?

 

I think after consuming kafka records from earliest to latest, the scan.startup.mode should no longer influence the kafka scan behave. So I suggest change the override to false.

Issue Links

• links to: GitHub Pull Request #43