Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1309

Parquet Java uses incorrect stats and dictionary filter properties

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.10.1
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      In SPARK-24251, we found that the changes to use HadoopReadOptions accidentally switched the properties that enable stats and dictionary filters. Both are enabled by default so it is unlikely that anyone will need to turn them off and there is an easy work-around, but we should fix the properties for 1.10.1. This doesn't affect the 1.8.x or 1.9.x releases (Spark 2.3.x is on 1.8.x).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gszadovszky Gabor Szadovszky
                Reporter:
                rdblue Ryan Blue
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: