Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14256

[Umbrella] Introduce FileSystemTableFactory with partitioned support

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: Table SQL / Planner
    • Labels:
      None

      Description

      Introduce FileSystemTableFactory to unify all file system connectors.

      More information in https://cwiki.apache.org/confluence/display/FLINK/FLIP-115%3A+Filesystem+connector+in+Table

        Attachments

          Issue Links

          1.
          Integrate hive to streaming file sink Sub-task Closed Jingsong Lee
          2.
          Integrate csv to FileSystemTableFactory Sub-task Closed Leonard Xu
          3.
          Integrate file system connector to streaming sink Sub-task Closed Jingsong Lee
          4.
          Introduce Row Csv InputFormat Sub-task Resolved Jingsong Lee
          5.
          Introduce Row Csv Encoder Sub-task Closed Leonard Xu
          6.
          Support partition for temporary table Sub-task Closed Jingsong Lee
          7.
          Add option to close shuffle when dynamic partition inserting Sub-task Closed Jingsong Lee  
          8.
          Support partition prune by getPartitions of source Sub-task Resolved Jingsong Lee
          9.
          Expose partitioned by grammar Sub-task Resolved Jingsong Lee
          10.
          Introduce FileSystemTableFactory, FileSystemTableSource, FileSystemTableSink Sub-task Resolved Jingsong Lee
          11.
          SingleDirectoryWriter should not produce file when no input record Sub-task Resolved Jingsong Lee
          12.
          Introduce table row write support for parquet writer Sub-task Resolved Jingsong Lee
          13.
          Integrate parquet to file system connector Sub-task Resolved Jingsong Lee
          14.
          Add docs for FileSystem connector Sub-task Closed Jingsong Lee  
          15.
          Add isBounded to TableSinkFactory#Context Sub-task Resolved Jingsong Lee
          16.
          Integrate json to file system connector Sub-task Closed Leonard Xu  
          17.
          Integrate orc to file system connector Sub-task Closed Jingsong Lee  
          18.
          sink.rolling-policy.time.interval default value should be bigger Sub-task Closed Jingsong Lee  
          19.
          Move FileSystemFormatFactory to table common Sub-task Closed Jingsong Lee  
          20.
          Create sql-jars for parquet and orc Sub-task Closed Jingsong Lee  
          21.
          Filesystem streaming sink support partition commit (success file) Sub-task Closed Jingsong Lee  
          22.
          Add project ITCase for partition table in filesystem connector Sub-task Closed Jingsong Lee  
          23.
          Integrate avro to file system connector Sub-task Closed Jingsong Lee  
          24.
          Add documentation for ORC format Sub-task Closed Jingsong Lee  
          25.
          Add documentation for Parquet format Sub-task Closed Jingsong Lee  
          26.
          Add filesystem documentation for Avro Sub-task Closed Jingsong Lee  

            Activity

              People

              • Assignee:
                lzljs3620320 Jingsong Lee
                Reporter:
                lzljs3620320 Jingsong Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m