[DRILL-2780] java.lang.IllegalStateException files open exceptions in drillbit.out

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Storage - Parquet
• Assignee: Steven Phillips
• Reporter: Ramana Inukonda Nagaraj
• Created: 14/Apr/15 04:09
• Updated: 14/May/15 19:07
• Resolved: 14/May/15 19:07

Description

There are many stacktraces like this in drillbit.out

Exception in thread "Thread-2" java.lang.IllegalStateException: Not all files opened using this FileSystem are closed. There are still [84] files open.
File '/drill/testdata/tpch100/parquet/lineitem/part-m-00045.parquet' opened at callstack:
        org.apache.drill.exec.store.parquet.columnreaders.PageReader.<init>(PageReader.java:105)
        org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.<init>(ColumnReader.java:88)
        org.apache.drill.exec.store.parquet.columnreaders.VarLengthColumn.<init>(VarLengthColumn.java:39)
        org.apache.drill.exec.store.parquet.columnreaders.VarLengthValuesColumn.<init>(VarLengthValuesColumn.java:43)
        org.apache.drill.exec.store.parquet.columnreaders.NullableVarLengthValuesColumn.<init>(NullableVarLengthValuesColumn.java:39)
        org.apache.drill.exec.store.parquet.columnreaders.VarLengthColumnReaders$NullableVarBinaryColumn.<init>(VarLengthColumnReaders.java:303)
        org.apache.drill.exec.store.parquet.columnreaders.ColumnReaderFactory.getReader(ColumnReaderFactory.java:176)
        org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.setup(ParquetRecordReader.java:303)
        org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:100)
        org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:157)
        org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:56)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
        org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:126)
        org.apache.drill.exec.store.parquet.ParquetRowGroupScan.accept(ParquetRowGroupScan.java:107)
        org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)
        org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator(AbstractPhysicalVisitor.java:214)
        org.apache.drill.exec.physical.config.IteratorValidator.accept(IteratorValidator.java:34)
        org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:74)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:62)
        org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:39)

We should not be getting any exceptions in drillbit.out.