Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21216

Write Parquet INT64 timestamp

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      This patch enables Hive to start writing int64 timestamps in Parquet.

      With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
      boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
      enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp

      The timestamp will have LocalDateTime semantics (not converted to UTC).
      Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
      Time unit (granularity) will be determined by the user. Default is milliseconds.

      Attachments

        1. HIVE-21216.1.patch
          64 kB
          Karen Coppage
        2. HIVE-21216.2.patch
          62 kB
          Karen Coppage
        3. HIVE-21216.3.patch
          62 kB
          Karen Coppage
        4. HIVE-21216.4.patch
          62 kB
          Karen Coppage

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            klcopp Karen Coppage Assign to me
            klcopp Karen Coppage
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment