[BEAM-5605] Support Portable SplittableDoFn for batch

• Type: New Feature
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.22.0
• Component/s: sdk-java-core
• Labels: Done portability
• Assignee: Luke Cwik
• Reporter: Scott Wegner
• Created: 01/Oct/18 23:02
• Updated: 06/Oct/20 21:54
• Resolved: 23/Apr/20 20:12

Description

Roll-up item tracking work towards supporting portable SplittableDoFn for batch

Issue Links

• fixes: BEAM-7430 Rename JavaReadViaImpulse to ReadViaImpulse
• is a child of: BEAM-2939 Fn API SDF support
• is a parent of: BEAM-5599 Convert all Java BoundedSources into SplittableDoFns
• is a parent of: BEAM-5600 Splitting for SplittableDoFn should be exposed within runner shared libraries
• is a parent of: BEAM-5602 Dataflow runner should support reporting progress for bounded SplittableDoFn
• is a parent of: BEAM-5604 Dataflow runner should support splitting bounded bundles for SplittableDoFn
• links to: GitHub Pull Request #8972
• links to: GitHub Pull Request #10483
• links to: GitHub Pull Request #10489
• links to: GitHub Pull Request #10490
• links to: GitHub Pull Request #10501
• links to: GitHub Pull Request #10535
• links to: GitHub Pull Request #10556
• links to: GitHub Pull Request #10576
• links to: GitHub Pull Request #10702
• links to: GitHub Pull Request #10893
• links to: GitHub Pull Request #10920
• links to: GitHub Pull Request #11414

Sub-Tasks

• SplittableDoFn dynamic rebalancing in Dataflow: Resolved
• SplittableDoFn: splitAtFraction() API for Java: Resolved
• Add support for splitting at fractions > 0 to org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker: Resolved