[BEAM-3365] The Java SDK Harness and runners-core-construction payloads should be consistent

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: runner-core, sdk-java-harness
• Labels: Clarified
• Assignee: Unassigned
• Reporter: Thomas Groh
• Created: 16/Dec/17 02:27
• Updated: 03/Jun/22 20:14

Description

Right now the Java SDK harness expects DoFnInfo instances within the payload of all DoFns it is provided, while the pipeline translation/ParDoTranslation/etc assumes that the ParDoTranslation channel will be used to extract useful values from the serialized form (potentially by converting to a ParDoLike). As such, a Java ParDo serialized with the Java ParDoTranslation class cannot be provided in that form to the Java SDK harness. Instead, that node has to be rewritten to include a DoFnInfo in the payload.

This seems unreasonable, and the two should be brought in line to use the same serialized representation.