[FLINK-26423] Integrate log store to StoreSink

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: table-store-0.1.0
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 01/Mar/22 12:06
• Updated: 08/Mar/22 06:48
• Resolved: 08/Mar/22 06:48

Description

StoreSink is a hybrid sink. We need:

• Introduce LocalCommitterOperator, kafka can only use the same produceId when committing, because starting multiple instances of a producer with a single produceId is not allowed by the kafka server. So we have to reuse the producer instance within the same process to commit.
• Committable can not be serialized, Kafka reuses producer instance in Committable. and where the producer will not be serialized.
• Integrate log store to StoreSink.

Issue Links

• links to: GitHub Pull Request #28