[BEAM-11391] Improve usability of Kafka source/sink for Nexmark

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: testing-nexmark
• Assignee: Sam Whittle
• Reporter: Sam Whittle
• Created: 03/Dec/20 12:38
• Updated: 21/Dec/20 14:56
• Resolved: 21/Dec/20 14:56

Description

Issues:

• remove unnecessary Key deserialization which fails if keys are not set on input
• no support for writing or reading events based upon create time, meaning watermarks are incorrect with historical events used by nexmark by default
• no support for launching pipelines from machines unable to communicate with kafka broker directly (due to need to determine # of partitions)

Issue Links

• links to: GitHub Pull Request #13472