Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5160

Memory leak in Parquet async reader when Snappy fails

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.9.0
    • Fix Version/s: 1.11.0
    • Component/s: None
    • Labels:
      None

      Description

      See the details in DRILL-5157. The Parquet async reader uses the Snappy library. If the call into Snappy fails (in my case, due to missing dependency management in the Drill pom.xml file), the Parquet reader fails (as it should.)

      Unfortunately, the Parquet reader leaks memory which cause IllegalStateException errors in production, assertion failures in unit tests.

      To reproduce this the easy way (to avoid the need to undo the fix for DRILL-5157 in Snappy dependencies), use Drill's exception injector in inject an exception at the line where we call into Snappy (See DRILL-5157 for details.)

      Set a breakpoint exception for IllegalStateException you will see the memory leak.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kkhatua Kunal Khatua
                Reporter:
                paul-rogers Paul Rogers
                Reviewer:
                Parth Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: