Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4336

Distributed Shell fails when used with the CapacityScheduler

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      DistributedShell attempts to get queue info without providing a queue name - which ends up in an NPE.

      1. MAPREDUCE-4336_rev2.patch
        2 kB
        Ahmed Radwan
      2. MAPREDUCE-4336.patch
        2 kB
        Ahmed Radwan

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1121 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1121/)
        MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1121 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1121/ ) MAPREDUCE-4336 . Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1088 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1088/)
        MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1088 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1088/ ) MAPREDUCE-4336 . Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2382 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2382/)
        MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2382 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2382/ ) MAPREDUCE-4336 . Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2452 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2452/)
        MAPREDUCE-4336. Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2452 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2452/ ) MAPREDUCE-4336 . Distributed Shell fails when used with the CapacityScheduler (ahmed via tucu) (Revision 1353625) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353625 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Ahmed. Committed to trunk and branch-2

        Show
        Alejandro Abdelnur added a comment - Thanks Ahmed. Committed to trunk and branch-2
        Hide
        Alejandro Abdelnur added a comment -

        +1. don't see the javac failure either.

        Show
        Alejandro Abdelnur added a comment - +1. don't see the javac failure either.
        Hide
        Ahmed Radwan added a comment -

        The javac -1 above seems weird. It successfully builds on my environment.

        Show
        Ahmed Radwan added a comment - The javac -1 above seems weird. It successfully builds on my environment.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12532614/MAPREDUCE-4336_rev2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javac. The patch appears to cause the build to fail.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2479//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532614/MAPREDUCE-4336_rev2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2479//console This message is automatically generated.
        Hide
        Ahmed Radwan added a comment -

        Thanks Siddharth! Here is the updated patch after rebase.

        Show
        Ahmed Radwan added a comment - Thanks Siddharth! Here is the updated patch after rebase.
        Hide
        Siddharth Seth added a comment -

        Patch looks good. The javadoc warnings don't seem related. It does need a rebase though.

        Show
        Siddharth Seth added a comment - Patch looks good. The javadoc warnings don't seem related. It does need a rebase though.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12532494/MAPREDUCE-4336.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 13 warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2473//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2473//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532494/MAPREDUCE-4336.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 13 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2473//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2473//console This message is automatically generated.
        Hide
        Ahmed Radwan added a comment -

        I have manually tested the patch by successfully submitting/running DS jobs on both the capacity and fifo schedulers.

        Show
        Ahmed Radwan added a comment - I have manually tested the patch by successfully submitting/running DS jobs on both the capacity and fifo schedulers.
        Hide
        Ahmed Radwan added a comment -

        The fix looks fairly straight forward: set the queue name for GetQueueInfoRequest, and also add "default" as the default queue name if not specified on the command line. I'll upload a patch now.

        Show
        Ahmed Radwan added a comment - The fix looks fairly straight forward: set the queue name for GetQueueInfoRequest, and also add "default" as the default queue name if not specified on the command line. I'll upload a patch now.

          People

          • Assignee:
            Ahmed Radwan
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development