[PARQUET-1963] DeprecatedParquetInputFormat in CombineFileInputFormat throw NPE when the first sub-split is empty

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: parquet-mr
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 19/Jan/21 00:10
• Updated: 23/Jun/24 03:32
• Resolved: 20/Jan/21 16:02

Description

A followup of PARQUET-1947, after the fix, when the first sub-split is empty in CombineFileInputFormat, there's a NPE:

Caused by: java.lang.NullPointerException
	at org.apache.parquet.hadoop.mapred.DeprecatedParquetInputFormat$RecordReaderWrapper.next(DeprecatedParquetInputFormat.java:154)
	at org.apache.parquet.hadoop.mapred.DeprecatedParquetInputFormat$RecordReaderWrapper.next(DeprecatedParquetInputFormat.java:73)
	at cascading.tap.hadoop.io.CombineFileRecordReaderWrapper.next(CombineFileRecordReaderWrapper.java:70)
	at org.apache.hadoop.mapred.lib.CombineFileRecordReader.next(CombineFileRecordReader.java:58)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:199)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:185)
	at cascading.tap.hadoop.util.MeasuredRecordReader.next(MeasuredRecordReader.java:61)
	at org.apache.parquet.cascading.ParquetTupleScheme.source(ParquetTupleScheme.java:160)
	at cascading.tuple.TupleEntrySchemeIterator.getNext(TupleEntrySchemeIterator.java:163)
	at cascading.tuple.TupleEntrySchemeIterator.hasNext(TupleEntrySchemeIterator.java:136)
	... 10 more

The reason is CombineFileInputFormat will use the result of createValue of the first sub-split as the value container. Since the first sub-split is empty, the value container is null.