[BEAM-7996] Add support for remaining data types in python RowCoder

• Type: New Feature
• Status: Open
• Priority: P1
• Resolution: Unresolved
• Component/s: cross-language, sdk-py-core
• Assignee: Unassigned
• Reporter: Brian Hulette
• Created: 16/Aug/19 22:34
• Updated: 04/Jun/22 13:38

Description

In the initial python RowCoder implementation we only added support for the data types that already had coders in the Python SDK. We should add support for the remaining data types that are not currently supported:

• INT8 (ByteCoder in Java)
• INT16 (BigEndianShortCoder in Java)
• FLOAT (FloatCoder in Java) (Note: doubles are supported, this is specifically for single-precision)
• -BOOLEAN (standard beam:coder:bool:v1, BooleanCoder in Java)-
• -BYTES (standard beam:coder:bytes:v1, ByteArrayCoder in Java)-
• -Map (MapCoder in Java)-

We might consider making those coders standard so they can be tested independently from RowCoder in standard_coders.yaml. Or, if we don't do that we should probably add a more robust testing framework for RowCoder itself, because it will be challenging to test all of these types as part of the RowCoder tests in standard_coders.yaml.

Issue Links

• is part of: BEAM-9621 Python SqlTransform follow-ups
• is related to: BEAM-8030 Make VarIntCoder overflow behavior consistent
• links to: GitHub Pull Request #12324
• links to: GitHub Pull Request #12426
• links to: GitHub Pull Request #12467
• links to: GitHub Pull Request #12530