Details
Major Description
When developing a simple Flink applications reading ORC files it crashes with NullPointerException when number of instances/executor threads is higher then the number of files because it is trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet initialized as there is no file for which it should have been opened. The issue is caused when
public void run(SourceContext<OUT> ctx) throws Exception { try { ... while (isRunning) { format.open(splitIterator.next()); ... } finally { format.close(); ... }
in file InputFormatSourceFunction.java which calls
public void close() throws IOException { // enforce sequential close() calls synchronized (CLOSE_MUTEX) { this.recordReader.close(); } }
from HadoopInputFormatBase.java.
As there is just this one implementation of the close() method it may be enough just to add a null check for the this.recordReader in there.
Attachments
Issue Links
- links to
