Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5316

job -list-attempt-ids command does not handle illegal task-state

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.0.4-alpha, 0.23.8
    • Fix Version/s: 0.23.9, 2.3.0
    • Component/s: client
    • Labels:
      None

      Description

      Courtesy : Phil Suh

      job -list-attempt-ids command should handle illegal argument for <task-state>
      the same way as <task-type>. Right now only illegal <task-type> is handle by
      an exception being thrown. Illegal <task-state> on the other hand does not
      throw exception. For example is a user mistype 'completed' as 'complete', they
      may wrongly think there are no completed tasks, instead of being notified of
      the illegal <task-state> that was used.

      1) illegal <task-type> handled.

      [philips@gwbl2003:4095 ~/svn/HadoopQEAutomation/branch-23]$
      /home/gs/gridre/yroot.theoden/share/hadoop/bin/mapred job -list-attempt-ids
      job_1345673924741_0086 map completed
      12/08/24 15:16:31 WARN conf.Configuration: mapred.used.genericoptionsparser is
      deprecated. Instead, use mapreduce.client.genericoptionsparser.used
      Exception in thread "main" java.lang.IllegalArgumentException: No enum const
      class org.apache.hadoop.mapreduce.TaskType.map
      at java.lang.Enum.valueOf(Enum.java:196)
      at org.apache.hadoop.mapreduce.TaskType.valueOf(TaskType.java:27)
      at org.apache.hadoop.mapreduce.tools.CLI.displayTasks(CLI.java:553)
      at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:309)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
      at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1242)

      1) illegal <task-state> not handled.

      /home/gs/gridre/yroot.theoden/share/hadoop/bin/mapred job -list-attempt-ids
      job_1345673924741_0086 MAP complete
      12/08/24 15:15:47 WARN conf.Configuration: mapred.used.genericoptionsparser is
      deprecated. Instead, use mapreduce.client.genericoptionsparser.used

      Also,we could make input task-state to be case-insensitive to be consistent with
      task-type behavior(MAPREDUCE-4019).

      1. MAPREDUCE-5316-1.txt
        5 kB
        Ashwin Shankar
      2. MAPREDUCE-5316-1-b23.txt
        5 kB
        Ashwin Shankar

        Activity

        Hide
        Ashwin Shankar added a comment -

        I've added code to validate task-state and print error/usage messages for invalid input.
        There is a small catch here- branch-23 didn't have any validation code for
        checking task-states,while trunk had one. However,in trunk, the validation code threw an exception. I've modified trunk as well to print error/usage message rather than throw an exception,so that we have consistent behavior across branches. Also,I've made task-state case-insensitive.

        Show
        Ashwin Shankar added a comment - I've added code to validate task-state and print error/usage messages for invalid input. There is a small catch here- branch-23 didn't have any validation code for checking task-states,while trunk had one. However,in trunk, the validation code threw an exception. I've modified trunk as well to print error/usage message rather than throw an exception,so that we have consistent behavior across branches. Also,I've made task-state case-insensitive.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12587886/MAPREDUCE-5316-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3773//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3773//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12587886/MAPREDUCE-5316-1.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3773//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3773//console This message is automatically generated.
        Hide
        Jason Lowe added a comment -

        +1, lgtm.

        Show
        Jason Lowe added a comment - +1, lgtm.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3967 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3967/)
        MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285)

        Result = SUCCESS
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3967 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3967/ ) MAPREDUCE-5316 . job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Hide
        Jason Lowe added a comment -

        Thanks, Ashwin! I committed this to trunk, branch-2, and branch-0.23.

        Show
        Jason Lowe added a comment - Thanks, Ashwin! I committed this to trunk, branch-2, and branch-0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #245 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/245/)
        MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285)

        Result = FAILURE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #245 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/245/ ) MAPREDUCE-5316 . job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #643 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/643/)
        MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494293)

        Result = SUCCESS
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494293
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #643 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/643/ ) MAPREDUCE-5316 . job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494293) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494293 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1435 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1435/)
        MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285)

        Result = FAILURE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1435 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1435/ ) MAPREDUCE-5316 . job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1462 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1462/)
        MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285)

        Result = FAILURE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1462 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1462/ ) MAPREDUCE-5316 . job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar (Revision 1494285) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1494285 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java

          People

          • Assignee:
            Ashwin Shankar
            Reporter:
            Ashwin Shankar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development