[BEAM-848] Shuffle input read-values to get maximum parallelism.

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Invalid
• Fix Version/s: 2.0.0
• Component/s: runner-spark
• Assignee: Aviem Zur
• Reporter: Amit Sela
• Created: 27/Oct/16 17:29
• Updated: 24/Jul/20 20:53
• Resolved: 03/Apr/17 16:38

Description

It would be wise to shuffle the read values after flatmap to increase parallelism in processing of the data.

Issue Links

• is related to: BEAM-1177 Input DStream "bundles" should be in serialized form and include relevant metadata.
• is related to: BEAM-1074 Set default-partitioner in SourceRDD.Unbounded.
• links to: GitHub Pull Request #2288

Sub-Tasks

• Sub-task?: Resolved