Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3557

Add workaround to create BIGINT stored as Kudu's UNIXTIME_MICROS

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: Kudu_Impala
    • Fix Version/s: Impala 2.9.0
    • Component/s: Backend
    • Labels:

      Description

      Kudu exposes a type called UNIXTIME_MICROS that is essentially an int64 and is intended to store microseconds since the unix epoch. Impala has a TIMESTAMP type which differs significantly (an INT96 storing something based on boost's "date" and "time_duration" classes).

      Users would like an Impala TIMESTAMP that works with Kudu, but this isn't feasible yet. For now, Impala will support creating Kudu tables with this underlying UNIXTIME_MICROS type but treating it otherwise as a BIGINT. In the future, Impala and Kudu will support a shared TIMESTAMP type. That work has yet to be planned.

      For now, Impala will provide syntax in CREATE TABLE for Kudu tables to create integer columns with a hint (comment containing a specific string) that indicates to Impala to create the underlying column as a Kudu UNIXTIME_MICROS type, but the column is an integer as far as Impala is concerned. Kudu’s UNIXTIME_MICROS type is just a int64, so it will be read and returned as such. All operations in Impala, e.g. describe table, select stmts, insert stmts, etc., treat this column as a BIGINT (because to Impala it is).

      Work items:
      Add support for hint in CREATE TABLE
      Add support for hint in ALTER TABLE

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              caseyc casey
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: