Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5628 Parquet support for additional valid decimal representations
  3. IMPALA-5542

Impala cannot scan Parquet decimal stored as int64_t/int32_t

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.10.0
    • Impala 3.1.0
    • Backend

    Description

      This is supported according to the Parquet spec (https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md#decimal) but wasn't widely used. For some reason Spark decided to start writing this out as the default (see SPARK-20297) so we will likely start seeing this at some point.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue IMPALA-2494 Impala Unable to scan a Decimal column stored as Bytes

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue IMPALA-2515 Impala rejects Parquet schemas where decimal fixed_len_byte_array columns have unnecessary padding bytes

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              boroknagyz Zoltán Borók-Nagy
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: