[BEAM-11300] Optimize Nexmark pipelines

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: testing-nexmark
• Assignee: Unassigned
• Reporter: Sam Whittle
• Created: 18/Nov/20 10:49
• Updated: 04/Jun/22 18:57

Description

Several of the nexmark queries can be improved to have better performance:

• Query3 uses stateful processing and a CogroupByKey which introduces an unnecessary shuffle compared to just using stateful processing. Additionally BagState is more appropriate to use than ValueState<List<>>
• Query5 uses a binary combiner when a custom combiner is more appropriate
• Query10 has excessive logging

Issue Links

• links to: GitHub Pull Request #13318