Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-27797

PythonTableUtils.getCollectionInputFormat cannot correctly handle None values

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      In `PythonTableUtils.getCollectionInputFormat` there are implementations like follows.
      This code can be found at https://github.com/apache/flink/blob/8488368b86a99a064446ca74e775b67ffff0b94a/flink-python/src/main/java/org/apache/flink/table/utils/python/PythonTableUtils.java#L515

      ```
      c -> {
      if (c.getClass() != byte[].class || dataType instanceof PickledByteArrayTypeInfo)

      { return c; }

      ```

      Here, the generated function did not check `c != null` before doing `c.getClass()`. which might cause that tables created through pyflink cannot parse it when values are `None`.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Zsigner EMing Zhou
            yunfengzhou Yunfeng Zhou
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment