Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8498

Write column index for floating types when NaN is not present

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.4.0
    • Component/s: None
    • Labels:
    • Epic Color:
      ghx-label-1

      Description

      IMPALA-7304 disabled column index writing for floating point columns until PARQUET-1222 is resolved.

      PARQUET-1222 is responsible for defining a total order for floating values, but the problematic values are only the NaNs. Therefore we can write the column index if NaNs are not present in the data. Parquet-MR also does this, following the principles in https://github.com/apache/parquet-format/blob/75eb7a7b84e6e62bfb09668b6d8d40b12597456e/src/main/thrift/parquet.thrift#L827-L834 

      Impala should follow this behavior, and also when storing zeroes, it should store -0.0 as minimum and +0.0 as maximum.

        Attachments

          Activity

            People

            • Assignee:
              norbertluksa Norbert Luksa
              Reporter:
              boroknagyz Zoltán Borók-Nagy
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: