Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      MAPREDUCE-6251 added support for retries to JobClient. However the MR CLI class doesn't use the JobClient. It would be useful to add support for retries to the CLI class as well.

      1. MAPREDUCE-6566.001.patch
        7 kB
        Varun Vasudev
      2. MAPREDUCE-6566.002.patch
        8 kB
        Varun Vasudev

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 2s trunk passed
          +1 compile 0m 22s trunk passed with JDK v1.8.0_66
          +1 compile 0m 26s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 33s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          -1 findbugs 1m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings.
          +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 28s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_66
          +1 javac 0m 23s the patch passed
          +1 compile 0m 25s the patch passed with JDK v1.7.0_85
          +1 javac 0m 25s the patch passed
          +1 checkstyle 0m 13s the patch passed
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 20s the patch passed
          +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_85
          -1 unit 2m 3s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66.
          -1 unit 2m 25s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 21s Patch generated 1 ASF License warnings.
          22m 15s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.tools.TestCLI
          JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.tools.TestCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776084/MAPREDUCE-6566.001.patch
          JIRA Issue MAPREDUCE-6566
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f606f15aca18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 01a641b
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 2s trunk passed +1 compile 0m 22s trunk passed with JDK v1.8.0_66 +1 compile 0m 26s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 33s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 14s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings. +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 28s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_66 +1 javac 0m 23s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_85 +1 javac 0m 25s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 20s the patch passed +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 26s the patch passed with JDK v1.7.0_85 -1 unit 2m 3s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66. -1 unit 2m 25s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 21s Patch generated 1 ASF License warnings. 22m 15s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.tools.TestCLI JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.tools.TestCLI Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776084/MAPREDUCE-6566.001.patch JIRA Issue MAPREDUCE-6566 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f606f15aca18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01a641b findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6203/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Uploaded a patch to fix the unit test errors.

          Show
          vvasudev Varun Vasudev added a comment - Uploaded a patch to fix the unit test errors.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 29s trunk passed
          +1 compile 0m 22s trunk passed with JDK v1.8.0_66
          +1 compile 0m 25s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 33s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          -1 findbugs 1m 15s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings.
          +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 23s the patch passed with JDK v1.8.0_66
          +1 javac 0m 23s the patch passed
          +1 compile 0m 25s the patch passed with JDK v1.7.0_85
          +1 javac 0m 25s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 24s the patch passed
          +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 27s the patch passed with JDK v1.7.0_85
          +1 unit 2m 5s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66.
          +1 unit 2m 26s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85.
          -1 asflicense 0m 20s Patch generated 1 ASF License warnings.
          22m 51s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776108/MAPREDUCE-6566.002.patch
          JIRA Issue MAPREDUCE-6566
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux db1b5ddfef4d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 01a641b
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 29s trunk passed +1 compile 0m 22s trunk passed with JDK v1.8.0_66 +1 compile 0m 25s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 33s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 15s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings. +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 29s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_66 +1 javac 0m 23s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_85 +1 javac 0m 25s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 24s the patch passed +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 27s the patch passed with JDK v1.7.0_85 +1 unit 2m 5s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66. +1 unit 2m 26s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85. -1 asflicense 0m 20s Patch generated 1 ASF License warnings. 22m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776108/MAPREDUCE-6566.002.patch JIRA Issue MAPREDUCE-6566 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux db1b5ddfef4d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01a641b findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6204/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Both the findbugs and ASF license issue are unrelated to the patch. Not sure why they're being flagged - the patch doesn't touch any of the files mentioned in the findbugs or ASF license report.

          Show
          vvasudev Varun Vasudev added a comment - Both the findbugs and ASF license issue are unrelated to the patch. Not sure why they're being flagged - the patch doesn't touch any of the files mentioned in the findbugs or ASF license report.
          Hide
          xgong Xuan Gong added a comment -

          +1 LGTM. Checking this in

          Show
          xgong Xuan Gong added a comment - +1 LGTM. Checking this in
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2/branch-2.7.2. Thanks, varun

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2/branch-2.7.2. Thanks, varun
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8937 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8937/)
          MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by (xgong: rev fc470840a0b1f5dc8b3b13f7ed99fd68ba728216)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8937 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8937/ ) MAPREDUCE-6566 . Add retry support to mapreduce CLI tool. Contributed by (xgong: rev fc470840a0b1f5dc8b3b13f7ed99fd68ba728216) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #674 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/674/)
          MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by (xgong: rev fc470840a0b1f5dc8b3b13f7ed99fd68ba728216)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #674 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/674/ ) MAPREDUCE-6566 . Add retry support to mapreduce CLI tool. Contributed by (xgong: rev fc470840a0b1f5dc8b3b13f7ed99fd68ba728216) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Xuan Gong, there are many major issues with how this patch got committed.

          • This is an improvement. It should never have been in 2.7.2. Why did you put it in 2.7.2?
          • Even if I give you that, it actually doesn't exist in 2.7.2 branch.
          • Neither does it exist in 2.8.0 branch.
          • What exactly happened with that trunk commit? You made too many CHANGES.txt modifications there.

          I'm reverting both the commits and reopening this JIRA to get this fixed correctly.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Xuan Gong , there are many major issues with how this patch got committed. This is an improvement. It should never have been in 2.7.2. Why did you put it in 2.7.2? Even if I give you that, it actually doesn't exist in 2.7.2 branch. Neither does it exist in 2.8.0 branch. What exactly happened with that trunk commit? You made too many CHANGES.txt modifications there. I'm reverting both the commits and reopening this JIRA to get this fixed correctly.
          Hide
          xgong Xuan Gong added a comment -

          Thanks, vinod.

          This is an improvement. It should never have been in 2.7.2. Why did you put it in 2.7.2?

          This is my fault. It should not be put into 2.7.2. Thanks for reverting the commit from branch-2.7

          Neither does it exist in 2.8.0 branch.

          Will commit the patch into branch-2.8

          What exactly happened with that trunk commit? You made too many CHANGES.txt modifications there.

          Not sure what happens previously, Will be more careful this time.

          Show
          xgong Xuan Gong added a comment - Thanks, vinod. This is an improvement. It should never have been in 2.7.2. Why did you put it in 2.7.2? This is my fault. It should not be put into 2.7.2. Thanks for reverting the commit from branch-2.7 Neither does it exist in 2.8.0 branch. Will commit the patch into branch-2.8 What exactly happened with that trunk commit? You made too many CHANGES.txt modifications there. Not sure what happens previously, Will be more careful this time.
          Hide
          xgong Xuan Gong added a comment -

          Committed into trunk/branch-2/branch-2.8.

          Show
          xgong Xuan Gong added a comment - Committed into trunk/branch-2/branch-2.8.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8980 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8980/)
          Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. (vinodkv: rev 60fe6d53d92c19c103219827fdb321780f1b33e5)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
            MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by (xgong: rev d4e766de931c1cc478561a211215c517ba1f341c)
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8980 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8980/ ) Revert " MAPREDUCE-6566 . Add retry support to mapreduce CLI tool. (vinodkv: rev 60fe6d53d92c19c103219827fdb321780f1b33e5) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java MAPREDUCE-6566 . Add retry support to mapreduce CLI tool. Contributed by (xgong: rev d4e766de931c1cc478561a211215c517ba1f341c) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
          Hide
          suda Akihiro Suda added a comment -

          The test is flaky, so I would like to open this issue again.

          -------------------------------------------------------------------------------
          Test set: org.apache.hadoop.mapreduce.tools.TestCLI
          -------------------------------------------------------------------------------
          Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.699 sec <<< FAILURE! - in org.apache.hadoop.mapreduce.tools.TestCLI
          testGetJob(org.apache.hadoop.mapreduce.tools.TestCLI)  Time elapsed: 0.025 sec  <<< FAILURE!
          java.lang.AssertionError: null
                  at org.junit.Assert.fail(Assert.java:86)
                  at org.junit.Assert.assertTrue(Assert.java:41)
                  at org.junit.Assert.assertTrue(Assert.java:52)
                  at org.apache.hadoop.mapreduce.tools.TestCLI.testGetJob(TestCLI.java:175)
          
          $ cat ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/surefire-reports/org.apache.hadoop.mapreduce.tools.TestCLI-output.txt
          Killed job job_1234654654_001
          The job job_1234654654_002 has already been killed.
          Could not mark the job job_1234654654_003 as killed, as it has already failed.
          Killed job job_1234654654_004
          Error: Invalid task-type: JOB_SETUP
          Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed
          Error: Invalid task-type: JOB_CLEANUP
          Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed
          Error: Invalid task-state: complete
          Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed
          

          https://github.com/apache/hadoop/blob/d4e766de931c1cc478561a211215c517ba1f341c/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java#L175

          Show
          suda Akihiro Suda added a comment - The test is flaky, so I would like to open this issue again. ------------------------------------------------------------------------------- Test set: org.apache.hadoop.mapreduce.tools.TestCLI ------------------------------------------------------------------------------- Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.699 sec <<< FAILURE! - in org.apache.hadoop.mapreduce.tools.TestCLI testGetJob(org.apache.hadoop.mapreduce.tools.TestCLI) Time elapsed: 0.025 sec <<< FAILURE! java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.mapreduce.tools.TestCLI.testGetJob(TestCLI.java:175) $ cat ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/surefire-reports/org.apache.hadoop.mapreduce.tools.TestCLI-output.txt Killed job job_1234654654_001 The job job_1234654654_002 has already been killed. Could not mark the job job_1234654654_003 as killed, as it has already failed. Killed job job_1234654654_004 Error: Invalid task-type: JOB_SETUP Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed Error: Invalid task-type: JOB_CLEANUP Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed Error: Invalid task-state: complete Usage: job [-list-attempt-ids <job-id> <task-type> <task-state>]. Valid values for <task-type> are MAP REDUCE. Valid values for <task-state> are pending, running, completed, failed, killed https://github.com/apache/hadoop/blob/d4e766de931c1cc478561a211215c517ba1f341c/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java#L175
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Akihiro Suda, this JIRA went on for a while, I am going to close this one as fixed. Please file a follow up ticket for the test flakiness. Tx.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Akihiro Suda , this JIRA went on for a while, I am going to close this one as fixed. Please file a follow up ticket for the test flakiness. Tx.

            People

            • Assignee:
              vvasudev Varun Vasudev
              Reporter:
              vvasudev Varun Vasudev
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development