Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-323

INT96 should be marked as deprecated

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • format-2.5.0
    • parquet-format
    • None

    Description

      As discussed in the mailing list, INT96 is only used to represent nanosec timestamp in Impala for some historical reasons, and should be deprecated. Since nanosec precision is rarely a real requirement, one possible and simple solution would be replacing INT96 with INT64 (TIMESTAMP_MILLIS) or INT64 (TIMESTAMP_MICROS).

      Several projects (Impala, Hive, Spark, ...) support INT96.
      We need a clear spec of the replacement and the path to deprecation.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1928 Interpret Parquet INT96 type as FIXED[12] AVRO Schema

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-904 Define INT96 ordering

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8824 Support Parquet time related logical types

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #86

          Web Link IMPALA-5049 - Switch Parquet timestamp format to INT64

          Activity

            People

              lv Lars Volker
              lian cheng Cheng Lian
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: