[BEAM-13298] Add "withResults()" for KafkaIO.write

• Type: Sub-task
• Status: In Progress
• Priority: P2
• Resolution: Unresolved
• Component/s: io-ideas, io-java-kafka
• Labels: KafkaIO stale-P2
• Environment: Production
• Flags: Important
• Assignee: Unassigned
• Reporter: Ranjan Dahal
• Created: 22/Nov/21 13:42
• Updated: 22/Nov/24 07:51

Description

I am looking at use case where we have to wait until the Kafka Write operation is completed before we can move forward with another transform. Currently, JdbcIO support withResults() which waits for the previous transform to complete as part of Wait.on(Signal) and moves on to the next. 

Similarly, it would be very beneficial to have this capability on KafkaIO (and others like PubSubIO, BigQueryIO etc). 

Issue Links

• relates to: BEAM-13302 KafkaIO: Add support for outputting a PCollection
• links to: GitHub Pull Request #16835