Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1309

Parquet Java uses incorrect stats and dictionary filter properties

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.11.0
    • parquet-mr
    • None

    Description

      In SPARK-24251, we found that the changes to use HadoopReadOptions accidentally switched the properties that enable stats and dictionary filters. Both are enabled by default so it is unlikely that anyone will need to turn them off and there is an easy work-around, but we should fix the properties for 1.10.1. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on 1.8.x).

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. PARQUET-1512 Release Parquet Java 1.10.1

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1318 Wrong configuration is passed from Hadoop config to Parquet options

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-26677 Incorrect results of not(eqNullSafe) when data read from Parquet file

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Web Link GitHub Pull Request #490

          Web Link PR

          Activity

            People

              gszadovszky Gabor Szadovszky
              rdblue Ryan Blue
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: