Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-2842

Parquet files with large file metadata sometimes fail to read in the FooterGather

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Storage - Parquet
    • Labels:
      None

      Description

      Parquet files with large footers could not be read. The length of the footer is written at the end of the file. To avoid excessive reads for smaller files, we read a reasonable amount of the end of the file that may contain the whole footer, with the actual exact length appearing at the end of the read. After checking the length we tried to read the remining portion ahead of what was already read and splice them together. The offset for where to put the bytes read first was off.

        Attachments

        1. 2842.patch
          8 kB
          Jason Altekruse

          Activity

            People

            • Assignee:
              jaltekruse Jason Altekruse
              Reporter:
              jaltekruse Jason Altekruse
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: