Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5518

HadoopInputFormat throws NPE when close() is called before open()

    Details

      Description

      When developing a simple Flink applications reading ORC files it crashes with NullPointerException when number of instances/executor threads is higher then the number of files because it is trying to close a HadoopInputFormat which is trying to close RecordReader which was not yet initialized as there is no file for which it should have been opened. The issue is caused when

      public void run(SourceContext<OUT> ctx) throws Exception {
          try {
      ...
              while (isRunning) {
      	    format.open(splitIterator.next());
      ...
          } finally {
      	format.close();
      ...
          }
      

      in file InputFormatSourceFunction.java which calls

      public void close() throws IOException {
          // enforce sequential close() calls
          synchronized (CLOSE_MUTEX) {
              this.recordReader.close();
          }
      }
      

      from HadoopInputFormatBase.java.

      As there is just this one implementation of the close() method it may be enough just to add a null check for the this.recordReader in there.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                havlikj Jakub Havlik
                Reporter:
                havlikj Jakub Havlik
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: