Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Target Version/s:

      Description

      The following mapred ant tests are failing. This started on December 22nd.

      [junit] Running org.apache.hadoop.mapred.TestTrackerBlacklistAcrossJobs
      [junit] Running org.apache.hadoop.mapred.TestMiniMRDFSSort
      [junit] Running org.apache.hadoop.mapred.TestBadRecords
      [junit] Running org.apache.hadoop.mapred.TestClusterMRNotification
      [junit] Running org.apache.hadoop.mapred.TestDebugScript
      [junit] Running org.apache.hadoop.mapred.TestJobCleanup
      [junit] Running org.apache.hadoop.mapred.TestJobClient
      [junit] Running org.apache.hadoop.mapred.TestJobHistory
      [junit] Running org.apache.hadoop.mapred.TestJobInProgressListener
      [junit] Running org.apache.hadoop.mapred.TestJobKillAndFail
      [junit] Running org.apache.hadoop.mapred.TestJvmReuse
      [junit] Running org.apache.hadoop.mapred.TestKillSubProcesses
      [junit] Running org.apache.hadoop.mapred.TestNodeRefresh
      [junit] Running org.apache.hadoop.mapred.TestSetupAndCleanupFailure
      [junit] Running org.apache.hadoop.mapred.TestTaskFail
      [junit] Running org.apache.hadoop.mapred.TestTaskOutputSize
      [junit] Running org.apache.hadoop.mapred.TestTaskTrackerSlotManagement
      [junit] Running org.apache.hadoop.mapreduce.TestMRJobClient
      [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob
      [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat

        Activity

        Hide
        Thomas Graves added a comment -

        The errors in the log seem to point that this was broke by MAPREDUCE-3349. I see the following error in a bunch of the logs:

        [junit] 2011-12-23 03:17:29,776 ERROR [IPC Server handler 5 on 60518] security.UserGroupInformation (UserGroupInformation.java:doAs(1160))

        • PriviledgedActionException as:hadoopqa (auth:SIMPLE) cause:java.io.IOException: java.lang.NullPointerException: null of string in field rackn
          ame of org.apache.hadoop.mapreduce.jobhistory.TaskAttemptUnsuccessfulCompletion of union in field event of org.apache.hadoop.mapreduce.jobhisto
          ry.Event
          [junit] 2011-12-23 03:17:29,777 INFO [IPC Server handler 5 on 60518] ipc.Server (Server.java:run(1498)) - IPC Server handler 5 on 60518, c
          all: heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@1e7c609, false, false, true, 41), rpc version=1, client version=31, methodsFingerPrin
          t=-746823657 from 127.0.0.1:34427, error:
          [junit] java.io.IOException: java.lang.NullPointerException: null of string in field rackname of org.apache.hadoop.mapreduce.jobhistory.Tas
          kAttemptUnsuccessfulCompletion of union in field event of org.apache.hadoop.mapreduce.jobhistory.Event
        Show
        Thomas Graves added a comment - The errors in the log seem to point that this was broke by MAPREDUCE-3349 . I see the following error in a bunch of the logs: [junit] 2011-12-23 03:17:29,776 ERROR [IPC Server handler 5 on 60518] security.UserGroupInformation (UserGroupInformation.java:doAs(1160)) PriviledgedActionException as:hadoopqa (auth:SIMPLE) cause:java.io.IOException: java.lang.NullPointerException: null of string in field rackn ame of org.apache.hadoop.mapreduce.jobhistory.TaskAttemptUnsuccessfulCompletion of union in field event of org.apache.hadoop.mapreduce.jobhisto ry.Event [junit] 2011-12-23 03:17:29,777 INFO [IPC Server handler 5 on 60518] ipc.Server (Server.java:run(1498)) - IPC Server handler 5 on 60518, c all: heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@1e7c609, false, false, true, 41), rpc version=1, client version=31, methodsFingerPrin t=-746823657 from 127.0.0.1:34427, error: [junit] java.io.IOException: java.lang.NullPointerException: null of string in field rackname of org.apache.hadoop.mapreduce.jobhistory.Tas kAttemptUnsuccessfulCompletion of union in field event of org.apache.hadoop.mapreduce.jobhistory.Event
        Hide
        Thomas Graves added a comment -

        The last 2 tests are failing for other reasons so splitting those off into separate jiras. These have actually been failing for a very long time.

        [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob
        [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat

        Show
        Thomas Graves added a comment - The last 2 tests are failing for other reasons so splitting those off into separate jiras. These have actually been failing for a very long time. [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDBJob [junit] Running org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12509462/MAPREDUCE-3615.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509462/MAPREDUCE-3615.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1544//console This message is automatically generated.
        Hide
        Thomas Graves added a comment -

        findbug warning was there previously. Also no tests included here because tests were already failing which caught this.

        Show
        Thomas Graves added a comment - findbug warning was there previously. Also no tests included here because tests were already failing which caught this.
        Hide
        Siddharth Seth added a comment -

        +1. Committed to trunk and branch-0.23. Thanks Thomas.

        Show
        Siddharth Seth added a comment - +1. Committed to trunk and branch-0.23. Thanks Thomas.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #332 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/332/)
        merge MAPREDUCE-3615 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #332 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/332/ ) merge MAPREDUCE-3615 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1572 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1572/)
        MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves)

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1572 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1572/ ) MAPREDUCE-3615 . Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #343 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/343/)
        merge MAPREDUCE-3615 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #343 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/343/ ) merge MAPREDUCE-3615 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1500 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1500/)
        MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves)

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1500 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1500/ ) MAPREDUCE-3615 . Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #354 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/354/)
        merge MAPREDUCE-3615 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #354 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/354/ ) merge MAPREDUCE-3615 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227742 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1520 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1520/)
        MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves)

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1520 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1520/ ) MAPREDUCE-3615 . Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #917 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/917/)
        MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves)

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #917 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/917/ ) MAPREDUCE-3615 . Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #950 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/950/)
        MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves)

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #950 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/950/ ) MAPREDUCE-3615 . Fix some ant test failures. (Contributed by Thomas Graves) sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227741 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development