[DRILL-8134] Regression: cannot query Parquet INT96 columns as timestamps

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.0
• Component/s: Storage - Parquet
• Labels: Regression
• Assignee: James Turton
• Reporter: James Turton
• Created: 15/Feb/22 10:19
• Updated: 17/Feb/22 17:03
• Resolved: 15/Feb/22 13:29

Description

Set store.parquet.reader.int96_as_timestamp = true and then query a file with an INT96 timestamp such as in the attachment.  INT96 columns get downcast to 64 bit timestamps, a fact that is ignored by some buggy new write buffer index positioning code that was merged in the 1.20 dev cycle.

result.tar.gz

 

Caused by: java.lang.NullPointerException:
        at org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:234)
        at org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:234)
        at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:298)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
        at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
        at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
        at org.apache.drill.exec.work.fragment.FragmentExecutor.lambda$run$0(FragmentExecutor.java:321)
        at .......(:0)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310)
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)