Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14826 Support vectorization for Parquet
  3. HIVE-18323

Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 2.4.0, 3.0.0
    • Vectorization
    • None

    Description

      CREATE TABLE `t1`(
        `ts` timestamp,
        `s1` string)
      STORED AS PARQUET;
      
      set hive.vectorized.execution.enabled=true;
      SELECT * from t1 SORT BY s1;
      

      This query will throw exception since timestamp is not supported here yet.

      Caused by: java.io.IOException: java.io.IOException: Unsupported type: optional int96 ts
              at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
              at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
              at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
              at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116)
      

      Attachments

        1. HIVE-18323.02.patch
          38 kB
          Vihang Karajgaonkar
        2. HIVE-18323.03.patch
          39 kB
          Vihang Karajgaonkar
        3. HIVE-18323.04.patch
          39 kB
          Vihang Karajgaonkar
        4. HIVE-18323.05.patch
          39 kB
          Vihang Karajgaonkar
        5. HIVE-18323.06.patch
          39 kB
          Vihang Karajgaonkar
        6. HIVE-18323.07.patch
          39 kB
          Vihang Karajgaonkar
        7. HIVE-18323.08.patch
          37 kB
          Vihang Karajgaonkar
        8. HIVE-18323.08-branch-2.patch
          37 kB
          Vihang Karajgaonkar
        9. HIVE-18323.1.patch
          3 kB
          Aihua Xu

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vihangk1 Vihang Karajgaonkar Assign to me
            aihuaxu Aihua Xu
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment