[PARQUET-2134] Incorrect type checking in HadoopStreams.wrap

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.8.3, 1.10.1, 1.11.2, 1.12.2
• Fix Version/s: 1.13.0
• Component/s: parquet-mr

Language

Java

• Assignee: Unassigned
• Reporter: Todd Gao
• Created: 09/Mar/22 03:11
• Updated: 13/Apr/23 23:47
• Resolved: 26/Mar/23 06:12

Description

The method HadoopStreams.wrap wraps an FSDataInputStream to a SeekableInputStream.

It checks whether the underlying stream of the passed FSDataInputStream implements ByteBufferReadable: if true, wraps the FSDataInputStream to H2SeekableInputStream; otherwise, wraps to H1SeekableInputStream.

In some cases, we may add another wrapper over FSDataInputStream. For example,

class CustomDataInputStream extends FSDataInputStream {
    public CustomDataInputStream(FSDataInputStream original) {
        super(original);
    }
}

When we create an FSDataInputStream, whose underlying stream does not implements ByteBufferReadable, and then creates a CustomDataInputStream with it. If we use HadoopStreams.wrap to create a SeekableInputStream, we may get an error like

java.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream

We can fix this by taking recursive checks over the underlying stream of FSDataInputStream.

Issue Links

• relates to: PARQUET-2151 Drop Hadoop 2 input stream reflection from parquet-hadoop
• relates to: HADOOP-18336 tag FSDataInputStream.getWrappedStream() @Public/@Stable