Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4794

DefaultSpeculator generates error messages on normal shutdown

    Details

      Description

      DefaultSpeculator can log the following error message on a normal shutdown of the ApplicationMaster:

      2012-11-13 01:35:31,841 ERROR [DefaultSpeculator background processing] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: Background thread returning, interrupted : java.lang.InterruptedException
      

      and in addition for some reason it logs the corresponding backtrace to stdout.

      Like the errors fixed in MAPREDUCE-4741, this error message in the syslog and backtrace on stdout can be confusing to users as to whether the job really succeeded.

        Issue Links

          Activity

          Hide
          Jerry Chen added a comment -

          I reproduced this problem and I will help to fix this.

          Show
          Jerry Chen added a comment - I reproduced this problem and I will help to fix this.
          Hide
          Jason Lowe added a comment -

          This has sat for a bit, so posting a quick patch. Hope that's OK with you, Jerry.

          I manually tested that without this patch the DefaultSpeculator always logs a stacktrace to stdout and an error to the syslog, and with the patch it doesn't log anything on normal shutdown.

          Show
          Jason Lowe added a comment - This has sat for a bit, so posting a quick patch. Hope that's OK with you, Jerry. I manually tested that without this patch the DefaultSpeculator always logs a stacktrace to stdout and an error to the syslog, and with the patch it doesn't log anything on normal shutdown.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12564953/MAPREDUCE-4794.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3240//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3240//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12564953/MAPREDUCE-4794.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3240//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3240//console This message is automatically generated.
          Hide
          Jonathan Eagles added a comment -

          +1. Thanks for the patch Jason.

          Show
          Jonathan Eagles added a comment - +1. Thanks for the patch Jason.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3406 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3406/)
          MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3406 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3406/ ) MAPREDUCE-4794 . DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/143/)
          MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/143/ ) MAPREDUCE-4794 . DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #541 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/541/)
          MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451832)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451832
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #541 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/541/ ) MAPREDUCE-4794 . DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451832) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451832 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1332 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1332/)
          MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1332 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1332/ ) MAPREDUCE-4794 . DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1360 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1360/)
          MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826)

          Result = SUCCESS
          jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1360 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1360/ ) MAPREDUCE-4794 . DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles) (Revision 1451826) Result = SUCCESS jeagles : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451826 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/speculate/DefaultSpeculator.java

            People

            • Assignee:
              Jason Lowe
              Reporter:
              Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development