Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6895

Job end notification not send due to YarnRuntimeException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.1, 2.8.0, 2.7.3
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: applicationmaster
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it caused job end notification not send.

      2017-05-24 12:14:02,165 WARN [Thread-693] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
      org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.nio.channels.ClosedChannelException
      at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
      at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
      at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
      at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
      at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
      at org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
      at org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
      at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
      at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
      at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
      at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
      at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
      Caused by: java.nio.channels.ClosedChannelException
      at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
      at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
      at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
      at java.io.DataOutputStream.write(DataOutputStream.java:107)
      at org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
      at org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
      at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
      at org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
      at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
      at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
      ... 11 more
      2017-05-24 12:14:02,165 INFO [Thread-693] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!

        Attachments

        1. MAPREDUCE-6895.002.patch
          5 kB
          yunjiong zhao
        2. MAPREDUCE-6895.001.patch
          4 kB
          yunjiong zhao

          Issue Links

            Activity

              People

              • Assignee:
                zhaoyunjiong yunjiong zhao
                Reporter:
                zhaoyunjiong yunjiong zhao
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: