Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14799 Improve batch sql and hive integrate performance milestone-2
  3. FLINK-11899

Introduce parquet ColumnarRow split reader

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Parquet ColumnarRow split reader is introduced to read parquet data in batches.

      When returning each row of data, instead of actually retrieving each field, we use BaseRow's abstraction to return a Columnar Row-like view.

      This will greatly improve the downstream filtered scenarios, so that there is no need to access redundant fields on the filtered data.

      Attachments

        Issue Links

        duplicates

        New Feature - A new feature of the product, which has yet to be developed. FLINK-15164 Introduce ParquetColumnarRowSplitReader to parquet format

        • Major - Major loss of function.
        • Closed
        is a child of

        Sub-task - The sub-task of the issue FLINK-11492 Support the full Blink SQL runtime

        • Major - Major loss of function.
        • Closed
        is related to

        Bug - A problem which impairs or prevents the functions of the product. FLINK-16032 Depends on core classifier hive-exec in hive connector

        • Not a Priority -
        • Open
        relates to

        Sub-task - The sub-task of the issue FLINK-16359 Introduce WritableVectors for abstract writing

        • Major - Major loss of function.
        • Resolved
        links to

        Web Link GitHub Pull Request #10922

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            lzljs3620320 Jingsong Lee
            lzljs3620320 Jingsong Lee
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment