Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21216

Write Parquet INT64 timestamp

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Hive
    • Labels:

      Description

      [WIP] NO PRECOMMIT TESTS

      This patch enables Hive to start writing int64 timestamps in Parquet.

      With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
      boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
      enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp

      The timestamp will have LocalDateTime semantics (not converted to UTC).
      Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
      Time unit (granularity) will be determined by the user. Default is milliseconds.

        Attachments

        1. HIVE-21216.1.patch
          64 kB
          Karen Coppage

          Issue Links

            Activity

              People

              • Assignee:
                klcopp Karen Coppage
                Reporter:
                klcopp Karen Coppage
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: