Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8120 Umbrella JIRA tracking Parquet improvements
  3. HIVE-8128

Improve Parquet Vectorization

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • parquet-branch
    • None
    • None

    Description

      We'll want to do is finish the vectorization work (e.g. VectorizedOrcSerde, VectorizedOrcSerde) which was partially done in HIVE-5998.

      As discussed in PARQUET-131, we will work out Hive POC based on the new Parquet vectorized API, and then finish the implementation after finilized.

      Attachments

        1. HIVE-8128.1-parquet.patch
          23 kB
          Dong Chen
        2. HIVE-8128.6-parquet.patch
          33 kB
          Ferdinand Xu
        3. HIVE-8128.6-parquet.patch
          33 kB
          Dong Chen
        4. HIVE-8128-parquet.patch.POC
          23 kB
          Dong Chen
        5. testParquetFile
          1.27 MB
          Dong Chen

        Issue Links

        is blocked by

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-8120 Umbrella JIRA tracking Parquet improvements

        • Major - Major loss of function.
        • Open
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HIVE-9235 Turn off Parquet Vectorization until all data types work: DECIMAL, DATE, TIMESTAMP, CHAR, and VARCHAR

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved
        relates to

        Sub-task - The sub-task of the issue HIVE-5998 Add vectorized reader for Parquet files

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. PARQUET-131 Vectorized Reader In Parquet

        • Major - Major loss of function.
        • Open
        links to

        Web Link Parquet Vectorization Proposal v1

        Web Link RB

        Web Link Review Board

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dongc Dong Chen Assign to me
            brocknoland Brock Noland
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment