[FLINK-30092] Improve Table API experience for Flink DOUBLE type

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Connectors / DynamoDB
• Assignee: Unassigned
• Reporter: Hong Liang Teoh
• Created: 18/Nov/22 16:13
• Updated: 19/Nov/22 11:06

Description

Currently, the DynamoDB sink's Table API allows users to input records with Flink type of `DOUBLE`. However, the limitations on Double on Flink/Java side [1] is more permissive than DynamoDB's limitations on the Number type [2].

 

Java Double limitations:

MIN_VALUE: −1.7976931348623157×10³⁰⁸ 

MAX_VALUE:  1.7976931348623157×10³⁰⁸

DynamoDB number type limitations:

MIN_VALUE: -9.9999999999999999999999999999999999999E+125 to -1E-130

MAX_VALUE: 1E-130 to 9.9999999999999999999999999999999999999E+125

 

This means that a user can write values of type `DOUBLE` into the DDB sink, as long as they are within the limitations of DynamoDB. Once they exceed the limitation, the record will fail to write.

 

We should consider improving this experience by adding a configuration to allow users to map DOUBLE to a specified DDB type (e.g. DDB String, DDB number) instead. We will be able to do this easily once FLINK-29267 is implemented.

 

 

 [1] https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#MAX_VALUE

 [2] https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#:~:text=Boolean%2C%20and%20null.-,Number,-Numbers%20can%20be 

Issue Links

• relates to: FLINK-29267 Support external type systems in DDL