[ORC-438] NPE in StringStatisticsImpl.merge()

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Sandeep More
• Reporter: Sandeep More
• Created: 21/Nov/18 18:43
• Updated: 04/Sep/19 02:00
• Resolved: 26/Nov/18 22:16

Description

StringStatisticsImpl.merge() does not test the conditions safely which can cause NPE e.g.

java.lang.NullPointerException
    at org.apache.hadoop.io.Text.encode(Text.java:443)
    at org.apache.hadoop.io.Text.set(Text.java:198)
    at org.apache.hadoop.io.Text.<init>(Text.java:88)
    at org.apache.orc.impl.ColumnStatisticsImpl$StringStatisticsImpl.merge(ColumnStatisticsImpl.java:610)
    at org.apache.orc.impl.writer.StringBaseTreeWriter.createRowIndexEntry(StringBaseTreeWriter.java:218)
    at org.apache.orc.impl.writer.StructTreeWriter.createRowIndexEntry(StructTreeWriter.java:106)
    at org.apache.orc.impl.WriterImpl.createRowIndexEntry(WriterImpl.java:435)
    at org.apache.orc.impl.WriterImpl.flushStripe(WriterImpl.java:441)
    at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:582)

Issue Links

• blocks: ORC-435 Ability to read stripes that are greater than 2GB
• is caused by: ORC-203 Modify the StringStatistics to trim minimum and maximum values
• links to: GitHub Pull Request #341