Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-751

DictionaryFilter patch broke column projection

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.9.0
    • 1.9.0, 1.8.2
    • parquet-mr
    • None

    Description

      The DictionaryFilter patch included a change to reuse a ParquetFileReader rather than creating new ones (and opening new file streams). But by reusing a reader, the projection columns are no longer set. The result is that all columns are read from disk, but only the requested columns are materialized. The fix is to set the requested schema on the file reader when it is fetched from the read context inside InternalParquetRecordReader.

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. PARQUET-392 Release Parquet-mr 1.9.0

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link PR #379

          Activity

            People

              rdblue Ryan Blue
              rdblue Ryan Blue
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: