Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5999

AMRMClientAsync will stop if any exceptions thrown on allocate call

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      Currently, for any exceptions thrown on the allocate call of AMRMClientAsync, it will stop both heartbeat thread and the callback handler thread, leaving AMRMClient in an unusable state. Caller has to instantiate a new AMRMClient.
      IMO, the threads should keep on running, it should be up to the caller whether to stop the AMRMClient or not.

                try {
                  response = client.allocate(progress);
                } catch (ApplicationAttemptNotFoundException e) {
                  handler.onShutdownRequest();
                  LOG.info("Shutdown requested. Stopping callback.");
                  return;
                } catch (Throwable ex) {
                  LOG.error("Exception on heartbeat", ex);
                  savedException = ex;
                  // interrupt handler thread in case it waiting on the queue
                  handlerThread.interrupt();
                  return;
                }
      

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 15s trunk passed
          +1 compile 0m 21s trunk passed
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 25s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 0m 30s trunk passed
          +1 javadoc 0m 16s trunk passed
          +1 mvninstall 0m 22s the patch passed
          +1 compile 0m 19s the patch passed
          +1 javac 0m 19s the patch passed
          +1 checkstyle 0m 13s the patch passed
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 16m 31s hadoop-yarn-client in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          30m 6s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-5999
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843155/YARN-5999.1.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 19f2bba3d5d3 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ada876c
          Default Java 1.8.0_111
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14291/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14291/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 15s trunk passed +1 compile 0m 21s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 16s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 16m 31s hadoop-yarn-client in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 30m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5999 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843155/YARN-5999.1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 19f2bba3d5d3 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ada876c Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14291/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14291/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          billie.rinaldi Billie Rinaldi added a comment -

          +1. The patch looks good to me. I tried it out with YARN-5996 and verified that the AM continues working after onError has been received. Thanks, Jian He!

          Show
          billie.rinaldi Billie Rinaldi added a comment - +1. The patch looks good to me. I tried it out with YARN-5996 and verified that the AM continues working after onError has been received. Thanks, Jian He !
          Hide
          xgong Xuan Gong added a comment -

          +1 LGTM. Checking this in

          Show
          xgong Xuan Gong added a comment - +1 LGTM. Checking this in
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10999 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10999/)
          YARN-5999. AMRMClientAsync will stop if any exceptions thrown on (xgong: rev 64a2d5be91a7f344aadf820481c542c967bc46a8)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/async/impl/TestAMRMClientAsync.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/async/impl/AMRMClientAsyncImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10999 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10999/ ) YARN-5999 . AMRMClientAsync will stop if any exceptions thrown on (xgong: rev 64a2d5be91a7f344aadf820481c542c967bc46a8) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/async/impl/TestAMRMClientAsync.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/async/impl/AMRMClientAsyncImpl.java
          Hide
          xgong Xuan Gong added a comment -

          Committed to trunk/branch-2. Thanks, Jian. And thanks, Billie Rinaldi for the review

          Show
          xgong Xuan Gong added a comment - Committed to trunk/branch-2. Thanks, Jian. And thanks, Billie Rinaldi for the review

            People

            • Assignee:
              jianhe Jian He
              Reporter:
              jianhe Jian He
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development