Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-686

Allow for Unsigned Statistics in Binary Type

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.9.0, 1.8.2
    • parquet-format
    • None

    Description

      BinaryStatistics currently only have a min/max, which are compared as signed byte[]. However, for real UTF8-friendly lexicographic comparison, e.g. for string columns, we would want to calculate the BinaryStatistics based off of a comparator that treats the bytes as unsigned.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1134 Release Parquet format 2.4.0

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. PARQUET-392 Release Parquet-mr 1.9.0

          • Major - Major loss of function.
          • Resolved
          causes

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1322 Statistics is not available for DECIMAL types

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-839 Min-max should be computed based on logical type

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-17213 Parquet String Pushdown for Non-Eq Comparisons Broken

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-2016 Reference column_order field from column indexes

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Format PR #42

          Web Link Format PR #46

          Web Link Format PR #55

          Web Link PR #362

          Web Link PR #367

          Activity

            People

              rdblue Ryan Blue
              andreweduffy Andrew Duffy
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: