Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3064

27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      unit test failure here: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/946/

      Test Result (27 failures / +27)

      org.apache.hadoop.mapred.TestCollect.testCollect
      org.apache.hadoop.mapred.TestComparators.testDefaultMRComparator
      org.apache.hadoop.mapred.TestComparators.testUserMRComparator
      org.apache.hadoop.mapred.TestComparators.testUserValueGroupingComparator
      org.apache.hadoop.mapred.TestComparators.testAllUserComparators
      org.apache.hadoop.mapred.TestFileOutputFormat.testCustomFile
      org.apache.hadoop.mapred.TestJavaSerialization.testMapReduceJob
      org.apache.hadoop.mapred.TestJavaSerialization.testWriteToSequencefile
      org.apache.hadoop.mapred.TestMapOutputType.testKeyMismatch
      org.apache.hadoop.mapred.TestMapOutputType.testValueMismatch
      org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
      org.apache.hadoop.mapred.TestMapRed.testMapred
      org.apache.hadoop.mapred.TestMapRed.testNullKeys
      org.apache.hadoop.mapred.TestMapRed.testCompression
      org.apache.hadoop.mapred.TestMapRed.testSmallInput
      org.apache.hadoop.mapred.TestMapRed.testBiggerInput
      org.apache.hadoop.mapreduce.TestMapCollection.testValLastByte
      org.apache.hadoop.mapreduce.TestMapCollection.testLargeRecords
      org.apache.hadoop.mapreduce.TestMapCollection.testSpillPer2B
      org.apache.hadoop.mapreduce.TestMapCollection.testZeroVal
      org.apache.hadoop.mapreduce.TestMapCollection.testSingleRecord
      org.apache.hadoop.mapreduce.TestMapCollection.testLowSpill
      org.apache.hadoop.mapreduce.TestMapCollection.testSplitMetaSpill
      org.apache.hadoop.mapreduce.TestMapCollection.testPostSpillMeta
      org.apache.hadoop.mapreduce.TestMapCollection.testLargeRecConcurrent
      org.apache.hadoop.mapreduce.TestMapCollection.testRandom
      org.apache.hadoop.mapreduce.TestMapCollection.testRandomCompress

      All of them have similar stack traces of:

      java.io.IOException: Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"
      at org.apache.hadoop.mapred.JobTrackerClientProtocolProvider.create(JobTrackerClientProtocolProvider.java:47)
      at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74)
      at org.apache.hadoop.mapred.JobClient.init(JobClient.java:459)
      at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:438)
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:777)
      at org.apache.hadoop.mapred.TestCollect.testCollect(TestCollect.java:133)

      1. MAPREDUCE-3064.1.patch
        5 kB
        Venu Gopala Rao
      2. MAPREDUCE-3064.patch
        4 kB
        Venu Gopala Rao

        Issue Links

          Activity

          Hide
          Thomas Graves added a comment -

          Looks like this was introduced in MAPREDUCE-2970, code added:

          +++ hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          @@ -43,20 +43,24 @@ public class JobTrackerClientProtocolProvider extends ClientProtocolProvider {
          String tracker = conf.get(JTConfig.JT_IPC_ADDRESS, "local");
          if (!"local".equals(tracker))

          { return createRPCProxy(JobTracker.getAddress(conf), conf); + }

          else

          { + throw new IOException("Invalid \"" + JTConfig.JT_IPC_ADDRESS + + "\" configuration value for JobTracker: \"" + + tracker + "\""); }
          • return null;
            }
          Show
          Thomas Graves added a comment - Looks like this was introduced in MAPREDUCE-2970 , code added: +++ hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java @@ -43,20 +43,24 @@ public class JobTrackerClientProtocolProvider extends ClientProtocolProvider { String tracker = conf.get(JTConfig.JT_IPC_ADDRESS, "local"); if (!"local".equals(tracker)) { return createRPCProxy(JobTracker.getAddress(conf), conf); + } else { + throw new IOException("Invalid \"" + JTConfig.JT_IPC_ADDRESS + + "\" configuration value for JobTracker: \"" + + tracker + "\""); } return null; }
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Oh no!

          We can fix those tests. Should be straight-forward. Worst case, we can let the "local" + jobtracker combination valid.

          I'd favour the former, though.

          Show
          Vinod Kumar Vavilapalli added a comment - Oh no! We can fix those tests. Should be straight-forward. Worst case, we can let the "local" + jobtracker combination valid. I'd favour the former, though.
          Hide
          Thomas Graves added a comment -

          Venu, would you be interested in/have time to look at this?

          Show
          Thomas Graves added a comment - Venu, would you be interested in/have time to look at this?
          Hide
          Venu Gopala Rao added a comment -

          Hey Thomas, Sorry, When I run these failing testcases didnot run I guess. Will fix and contribute the patch.

          Show
          Venu Gopala Rao added a comment - Hey Thomas, Sorry, When I run these failing testcases didnot run I guess. Will fix and contribute the patch.
          Hide
          Arun C Murthy added a comment -

          Thanks Venu!

          Show
          Arun C Murthy added a comment - Thanks Venu!
          Hide
          Vinod Kumar Vavilapalli added a comment -

          When I run these failing testcases didnot run I guess.

          Venu, these test are in the old mapreduce code, which is an ant build.
          So, to run these tests, you will have to do the following in hadoop-mapreduce-project:

          • mvn clean install
          • ant veryclean test -Dresolvers=internal
            These tests are going to take a long time, so I'd advise you not to do it on your dev machine
          Show
          Vinod Kumar Vavilapalli added a comment - When I run these failing testcases didnot run I guess. Venu, these test are in the old mapreduce code, which is an ant build. So, to run these tests, you will have to do the following in hadoop-mapreduce-project: mvn clean install ant veryclean test -Dresolvers=internal These tests are going to take a long time, so I'd advise you not to do it on your dev machine
          Hide
          Mahadev konar added a comment -

          Vinod/Venu,
          You dont have to run the full test suite. Running run-commit-test should be good enough:

          • mvn clean install
          • ant veryclean run-commit-test -Dresolvers=internal

          ant test is a little unstable with some failing tests.

          Show
          Mahadev konar added a comment - Vinod/Venu, You dont have to run the full test suite. Running run-commit-test should be good enough: mvn clean install ant veryclean run-commit-test -Dresolvers=internal ant test is a little unstable with some failing tests.
          Hide
          Devaraj K added a comment -

          Mahdev,
          Can we integrate these testcases to Hadoop QA.

          • ant veryclean run-commit-test -Dresolvers=internal

          Here Hadoop QA is giving +1 and testcases are failing in Hadoop-Mapreduce-trunk-Commit may not be good.

          Show
          Devaraj K added a comment - Mahdev, Can we integrate these testcases to Hadoop QA. ant veryclean run-commit-test -Dresolvers=internal Here Hadoop QA is giving +1 and testcases are failing in Hadoop-Mapreduce-trunk-Commit may not be good.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Can we integrate these testcases to Hadoop QA.

          Most of these test JT/TT etc. and adding all of them will only slow-down the commit build. IMO, we can live with the separation in the interim.

          Show
          Vinod Kumar Vavilapalli added a comment - Can we integrate these testcases to Hadoop QA. Most of these test JT/TT etc. and adding all of them will only slow-down the commit build. IMO, we can live with the separation in the interim.
          Hide
          Venu Gopala Rao added a comment -

          Execute the run-commit-tests target. For some reason the execution is very very slow. All the tests that failed yesterday are passing now.

          [junit] Running org.apache.hadoop.mapred.TestCollect
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.379 sec
          [junit] Running org.apache.hadoop.mapred.TestCommandLineJobSubmission
          [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 13.169 sec
          [junit] Test org.apache.hadoop.mapred.TestCommandLineJobSubmission FAILED
          [junit] Running org.apache.hadoop.mapred.TestComparators
          [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 10.991 sec
          [junit] Running org.apache.hadoop.mapred.TestCounters
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.325 sec
          [junit] Running org.apache.hadoop.mapred.TestFileInputFormat
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 35.604 sec
          [junit] Running org.apache.hadoop.mapred.TestFileInputFormatPathFilter
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.805 sec
          [junit] Running org.apache.hadoop.mapred.TestFileOutputCommitter
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.441 sec
          [junit] Running org.apache.hadoop.mapred.TestFileOutputFormat
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.056 sec
          [junit] Running org.apache.hadoop.mapred.TestGetSplitHosts
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.549 sec
          [junit] Running org.apache.hadoop.mapred.TestIFileStreams
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.056 sec
          [junit] Running org.apache.hadoop.mapred.TestIndexCache
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 4.732 sec
          [junit] Running org.apache.hadoop.mapred.TestJavaSerialization
          [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 6.373 sec
          [junit] Running org.apache.hadoop.mapred.TestJobHistoryParsing
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.977 sec
          [junit] Running org.apache.hadoop.mapred.TestJobInProgress
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.508 sec
          [junit] Running org.apache.hadoop.mapred.TestJobQueueTaskScheduler
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 5.133 sec
          [junit] Running org.apache.hadoop.mapred.TestKillCompletedJob
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.442 sec
          [junit] Running org.apache.hadoop.mapred.TestLostTracker
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 67.895 sec
          [junit] Running org.apache.hadoop.mapred.TestMapOutputType
          [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.69 sec
          [junit] Running org.apache.hadoop.mapred.TestMapRed
          [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 59.254 sec

          TestCommandLineJobSubmission failure in my machine is not due to my change. Also kept the build running for all testcases to night along with the run-commit-test.

          Show
          Venu Gopala Rao added a comment - Execute the run-commit-tests target. For some reason the execution is very very slow. All the tests that failed yesterday are passing now. [junit] Running org.apache.hadoop.mapred.TestCollect [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.379 sec [junit] Running org.apache.hadoop.mapred.TestCommandLineJobSubmission [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 13.169 sec [junit] Test org.apache.hadoop.mapred.TestCommandLineJobSubmission FAILED [junit] Running org.apache.hadoop.mapred.TestComparators [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 10.991 sec [junit] Running org.apache.hadoop.mapred.TestCounters [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.325 sec [junit] Running org.apache.hadoop.mapred.TestFileInputFormat [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 35.604 sec [junit] Running org.apache.hadoop.mapred.TestFileInputFormatPathFilter [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.805 sec [junit] Running org.apache.hadoop.mapred.TestFileOutputCommitter [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.441 sec [junit] Running org.apache.hadoop.mapred.TestFileOutputFormat [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.056 sec [junit] Running org.apache.hadoop.mapred.TestGetSplitHosts [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.549 sec [junit] Running org.apache.hadoop.mapred.TestIFileStreams [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.056 sec [junit] Running org.apache.hadoop.mapred.TestIndexCache [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 4.732 sec [junit] Running org.apache.hadoop.mapred.TestJavaSerialization [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 6.373 sec [junit] Running org.apache.hadoop.mapred.TestJobHistoryParsing [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 8.977 sec [junit] Running org.apache.hadoop.mapred.TestJobInProgress [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.508 sec [junit] Running org.apache.hadoop.mapred.TestJobQueueTaskScheduler [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 5.133 sec [junit] Running org.apache.hadoop.mapred.TestKillCompletedJob [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.442 sec [junit] Running org.apache.hadoop.mapred.TestLostTracker [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 67.895 sec [junit] Running org.apache.hadoop.mapred.TestMapOutputType [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 8.69 sec [junit] Running org.apache.hadoop.mapred.TestMapRed [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 59.254 sec TestCommandLineJobSubmission failure in my machine is not due to my change. Also kept the build running for all testcases to night along with the run-commit-test.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496186/MAPREDUCE-3064.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/832//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/832//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496186/MAPREDUCE-3064.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/832//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/832//console This message is automatically generated.
          Hide
          Mahadev konar added a comment -

          @venu,
          Looks like you are fixing the tests to make them pass. In that case we'll have to fix all the tests that fail with ant test -Dresolvers=internal. Is there a fix that can be done without changing the test cases (meaning something in the core code)?

          Show
          Mahadev konar added a comment - @venu, Looks like you are fixing the tests to make them pass. In that case we'll have to fix all the tests that fail with ant test -Dresolvers=internal. Is there a fix that can be done without changing the test cases (meaning something in the core code)?
          Hide
          Mahadev konar added a comment -

          Just read your comment on running the other tests. Do comment with a full ant test result. I am ok with the change as long as none of the tests in ant test -Dresolvers=internal are failing due to MAPREDUCE-2970.

          Show
          Mahadev konar added a comment - Just read your comment on running the other tests. Do comment with a full ant test result. I am ok with the change as long as none of the tests in ant test -Dresolvers=internal are failing due to MAPREDUCE-2970 .
          Hide
          Venu Gopala Rao added a comment -

          Executed all test cases now and rebased the patch for all failing cases. One observation is that in my environment I get around 10 failures (mostly timeout) and few Job failures. How ever they are not related to this problem.

          Show
          Venu Gopala Rao added a comment - Executed all test cases now and rebased the patch for all failing cases. One observation is that in my environment I get around 10 failures (mostly timeout) and few Job failures. How ever they are not related to this problem.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496377/MAPREDUCE-3064.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/850//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/850//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496377/MAPREDUCE-3064.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/850//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/850//console This message is automatically generated.
          Hide
          Mahadev konar added a comment -

          Venu,
          Is run-commit-test passing? Looks like in the above comments you mentioned that TestCommandLineJobSubmission is failing?

          Show
          Mahadev konar added a comment - Venu, Is run-commit-test passing? Looks like in the above comments you mentioned that TestCommandLineJobSubmission is failing?
          Hide
          Venu Gopala Rao added a comment -

          Yes MahaDev all Testcases are passing in run-commit-test.

          Some test cases in run-test-mapred are failing. But they are not related to the MAPREDUCE-2970.

          Show
          Venu Gopala Rao added a comment - Yes MahaDev all Testcases are passing in run-commit-test. Some test cases in run-test-mapred are failing. But they are not related to the MAPREDUCE-2970 .
          Hide
          Mahadev konar added a comment -

          Thanks a lot Venu. I just committed this to trunk and 0.23. Would you mind opening a jira for the 10 tests that are failing anyways? We should track them and disable if needed for a clean build.

          Show
          Mahadev konar added a comment - Thanks a lot Venu. I just committed this to trunk and 0.23. Would you mind opening a jira for the 10 tests that are failing anyways? We should track them and disable if needed for a clean build.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #953 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/953/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #953 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/953/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1031 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1031/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1031 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1031/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #971 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/971/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #971 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/971/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Venu Gopala Rao added a comment -

          Hi Mahadev, After committing this the Jenkins is failing with a Compilation error MRConfig does not have CLASSIC_FRAMEWORK_NAME. But in the code trunk I see the Constant is there. Also my local builds are passing. Am I missing some thing?

          
          [javac] Compiling 48 source files to /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk-Commit/trunk/hadoop-mapreduce-project/build/classes
              [javac] /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk-Commit/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java:40: cannot find symbol
              [javac] symbol  : variable CLASSIC_FRAMEWORK_NAME
              [javac] location: interface org.apache.hadoop.mapreduce.MRConfig
              [javac]     if (!MRConfig.CLASSIC_FRAMEWORK_NAME.equals(framework)) {
              [javac]                  ^
              [javac] Note: Some input files use or override a deprecated API.
              [javac] Note: Recompile with -Xlint:deprecation for details.
              [javac] 1 error
          
          
          Show
          Venu Gopala Rao added a comment - Hi Mahadev, After committing this the Jenkins is failing with a Compilation error MRConfig does not have CLASSIC_FRAMEWORK_NAME. But in the code trunk I see the Constant is there. Also my local builds are passing. Am I missing some thing? [javac] Compiling 48 source files to /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk-Commit/trunk/hadoop-mapreduce-project/build/classes [javac] /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk-Commit/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java:40: cannot find symbol [javac] symbol : variable CLASSIC_FRAMEWORK_NAME [javac] location: interface org.apache.hadoop.mapreduce.MRConfig [javac] if (!MRConfig.CLASSIC_FRAMEWORK_NAME.equals(framework)) { [javac] ^ [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] 1 error
          Hide
          Venu Gopala Rao added a comment -

          Just took a clean workspace and built every thing from scratch. My local build is passing. Is there any possibility that Jenkins has an outdated jar in classpath?

          Show
          Venu Gopala Rao added a comment - Just took a clean workspace and built every thing from scratch. My local build is passing. Is there any possibility that Jenkins has an outdated jar in classpath?
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #812 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/812/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #812 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/812/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #21 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/21/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) - Merging r1175449 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175450
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #21 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/21/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) - Merging r1175449 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175450 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #27 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/27/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) - Merging r1175449 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175450
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #27 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/27/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) - Merging r1175449 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175450 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #842 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/842/)
          MAPREDUCE-3064. 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #842 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/842/ ) MAPREDUCE-3064 . 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local (Venu Gopala Rao via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1175449 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobTrackerClientProtocolProvider.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/QueueManagerTestUtils.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRClasspath.java /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSpecialCharactersInOutputPath.java
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This patch broke the ant build for MRV1, I cannot compile a test. (ant veryclean all-jars -Dresolvers=internal)

          Show
          Vinod Kumar Vavilapalli added a comment - This patch broke the ant build for MRV1, I cannot compile a test. (ant veryclean all-jars -Dresolvers=internal)
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Never mind, ran into the stale jars issue.

          Show
          Vinod Kumar Vavilapalli added a comment - Never mind, ran into the stale jars issue.
          Hide
          Venu Gopala Rao added a comment -

          Vinod, any idea why Jenkins is getting compilation issue?

          Show
          Venu Gopala Rao added a comment - Vinod, any idea why Jenkins is getting compilation issue?
          Hide
          Mahadev konar added a comment -

          @venu,
          Jenkins is running into some stale jars with ivy cache. Ill fix that sometime today.

          Show
          Mahadev konar added a comment - @venu, Jenkins is running into some stale jars with ivy cache. Ill fix that sometime today.
          Hide
          Mahadev konar added a comment -

          @venu,
          Looks like the tests are still failing:

          https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/978/

          Can you please take a look?

          thanks

          Show
          Mahadev konar added a comment - @venu, Looks like the tests are still failing: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/978/ Can you please take a look? thanks
          Hide
          Mahadev konar added a comment -

          Reopening the issue, we are still having issues with the build:

          https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/983/

          Venu, can you please take a look?

          Show
          Mahadev konar added a comment - Reopening the issue, we are still having issues with the build: https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/983/ Venu, can you please take a look?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I checked both the builds referred to by Mahadev. They both point to the issues I am running into with HADOOP-7542.

          Closing this ticket. Let's open new ones if we run into issues again.

          Show
          Vinod Kumar Vavilapalli added a comment - I checked both the builds referred to by Mahadev. They both point to the issues I am running into with HADOOP-7542 . Closing this ticket. Let's open new ones if we run into issues again.
          Hide
          Mahadev konar added a comment -

          Vinod,
          Looks like HADOOP-7542 was reverted but the tests are still failing. You sure it was HADOOP-7542?

          Show
          Mahadev konar added a comment - Vinod, Looks like HADOOP-7542 was reverted but the tests are still failing. You sure it was HADOOP-7542 ?
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I suspected that at that point of time. Trunk commits are failing are because of a different bug. I am opening a separate ticket for that.

          Show
          Vinod Kumar Vavilapalli added a comment - I suspected that at that point of time. Trunk commits are failing are because of a different bug. I am opening a separate ticket for that.

            People

            • Assignee:
              Venu Gopala Rao
              Reporter:
              Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development