Hadoop YARN
  1. Hadoop YARN
  2. YARN-277

Use AMRMClient in DistributedShell to exemplify the approach

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3-alpha
    • Component/s: None
    • Labels:
      None
    1. YARN-277.2.patch
      54 kB
      Hitesh Shah
    2. YARN-277.1.patch
      27 kB
      Bikas Saha

      Issue Links

        Activity

        Hitesh Shah made changes -
        Link This issue duplicates YARN-130 [ YARN-130 ]
        Hitesh Shah made changes -
        Link This issue duplicates YARN-130 [ YARN-130 ]
        Hitesh Shah made changes -
        Link This issue duplicates YARN-130 [ YARN-130 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1322 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1322/)
        YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156)

        Result = SUCCESS
        hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1322 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1322/ ) YARN-277 . Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1294 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1294/)
        YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156)

        Result = FAILURE
        hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1294 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1294/ ) YARN-277 . Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156) Result = FAILURE hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #105 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/105/)
        YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156)

        Result = SUCCESS
        hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #105 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/105/ ) YARN-277 . Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3271 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3271/)
        YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156)

        Result = SUCCESS
        hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3271 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3271/ ) YARN-277 . Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha (Revision 1437156) Result = SUCCESS hitesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1437156 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
        Hitesh Shah made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.0.3-alpha [ 12323272 ]
        Fix Version/s 3.0.0 [ 12323268 ]
        Resolution Fixed [ 1 ]
        Hide
        Hitesh Shah added a comment -

        @Bikas, thanks for the patch. @Sid, thanks for the review. Committed to trunk and branch-2.

        Show
        Hitesh Shah added a comment - @Bikas, thanks for the patch. @Sid, thanks for the review. Committed to trunk and branch-2.
        Hide
        Siddharth Seth added a comment -

        +1. lgtm.

        Show
        Siddharth Seth added a comment - +1. lgtm.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12563634/YARN-277.2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/321//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/321//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12563634/YARN-277.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/321//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/321//console This message is automatically generated.
        Hitesh Shah made changes -
        Attachment YARN-277.2.patch [ 12563634 ]
        Hide
        Hitesh Shah added a comment -

        Original changes looked fine.

        However, I modified patch slightly to make code use YarnConfiguration correctly so as to make code work with non-default values.

        Also, removed unused imports and cleaned up lines extending over 80 chars and also whitespaces at end of lines.

        Show
        Hitesh Shah added a comment - Original changes looked fine. However, I modified patch slightly to make code use YarnConfiguration correctly so as to make code work with non-default values. Also, removed unused imports and cleaned up lines extending over 80 chars and also whitespaces at end of lines.
        Hide
        Bikas Saha added a comment -

        This is a refactoring of code in DistShell and so the existing test DistShell covers the change.

        Show
        Bikas Saha added a comment - This is a refactoring of code in DistShell and so the existing test DistShell covers the change.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12563182/YARN-277.1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/274//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/274//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12563182/YARN-277.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/274//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/274//console This message is automatically generated.
        Bikas Saha made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Bikas Saha made changes -
        Attachment YARN-277.1.patch [ 12563182 ]
        Hide
        Bikas Saha added a comment -

        Attaching patch that uses AMRMClient in DistShell

        Show
        Bikas Saha added a comment - Attaching patch that uses AMRMClient in DistShell
        Bikas Saha made changes -
        Field Original Value New Value
        Link This issue is blocked by YARN-103 [ YARN-103 ]
        Bikas Saha created issue -

          People

          • Assignee:
            Bikas Saha
            Reporter:
            Bikas Saha
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development