[FLINK-34203] PyFlink `get_field_data_types()` supports only `precision = 3` for `TimestampType`

• Type: Technical Debt
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: API / Python
• Assignee: Unassigned
• Reporter: Mehmet Aktas
• Created: 22/Jan/24 18:51
• Updated: 22/Jan/24 18:51

Description

PyFlink schema's `get_field_data_types()` set `precision` to 3 for `TIMESTAMP` regardless of the input precision – copied the relevant pieces of code below. There does not seem to be any reason for this implicit assumption. It led to an error in our project, forcing us to implement a wrapper around `get_field_data_types()`.

 

pyflink/table/table_schema.py: get_field_data_types():

    def get_field_data_types(self) -> List[DataType]:
        """
        Returns all field data types as a list.        :return: A list of all field data types.
        """
        return [_from_java_data_type(item) for item in self._j_table_schema.getFieldDataTypes()] 

pyflink/table/types.py: _from_java_data_type():

def _from_java_data_type(j_data_type):
    """
    Converts Java DataType to Python DataType.
    """
    ...
    if is_instance_of(j_data_type, gateway.jvm.AtomicDataType):
        logical_type = j_data_type.getLogicalType()     
    ...

        elif is_instance_of(logical_type, gateway.jvm.TimestampType):
            data_type = DataTypes.TIMESTAMP(precision=3, nullable=logical_type.isNullable())
    ...