Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6001

Improve moveApplicationQueues command line

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As an example, below command is used to move an application across queue.

      ./yarn application -movetoqueue  application_1479894790219_0002 -queue default

      Inline with other application's attribute modification features such as updateTimeout or priority, movetoqueue cli command lacks unification and more complex or error prone.

      Suggesting below cli command which can be used.

      ./yarn application -appId application_1479894790219_0002 -move default
      

      This is inline with other commands such as

      ./yarn application -appId  application_1479894790219_0002 -updatePriority 8 
      ./yarn application -appId  application_1479894790219_0002 -updateLifetime 10
      

      Old movetoqueue command could be still kept, but we can mark it as deprecated and mention in help message.

      1. YARN-6001.0002.patch
        9 kB
        Sunil G
      2. YARN-6001.0001.patch
        8 kB
        Sunil G

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11047 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11047/)
        YARN-6001. Improve moveApplicationQueues command line. Contributed by (rohithsharmaks: rev e297be74ed4c4334bf48dd01d3b10e302147041d)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11047 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11047/ ) YARN-6001 . Improve moveApplicationQueues command line. Contributed by (rohithsharmaks: rev e297be74ed4c4334bf48dd01d3b10e302147041d) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java
        Hide
        sunilg Sunil G added a comment -

        Thanks Rohith Sharma K S for review and commit and thanks Yufei Gu for additional review! Appreciate it.

        Show
        sunilg Sunil G added a comment - Thanks Rohith Sharma K S for review and commit and thanks Yufei Gu for additional review! Appreciate it.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        committed to trunk/branch-2.. thanks Sunil for your contributions.

        Show
        rohithsharma Rohith Sharma K S added a comment - committed to trunk/branch-2.. thanks Sunil for your contributions.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 56s trunk passed
        +1 compile 0m 23s trunk passed
        +1 checkstyle 0m 19s trunk passed
        +1 mvnsite 0m 27s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 33s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 21s the patch passed
        +1 compile 0m 21s the patch passed
        +1 javac 0m 21s the patch passed
        -0 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 5 new + 150 unchanged - 1 fixed = 155 total (was 151)
        +1 mvnsite 0m 23s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 45s the patch passed
        +1 javadoc 0m 14s the patch passed
        +1 unit 16m 12s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        37m 48s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6001
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844048/YARN-6001.0002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 7d1b6dd3cb49 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ea54752
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14467/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14467/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14467/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 56s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed -0 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 5 new + 150 unchanged - 1 fixed = 155 total (was 151) +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 45s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 16m 12s hadoop-yarn-client in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 37m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6001 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844048/YARN-6001.0002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7d1b6dd3cb49 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ea54752 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14467/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14467/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14467/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        I have triggered build manually, lets wait for Jenkins to run.

        Show
        rohithsharma Rohith Sharma K S added a comment - I have triggered build manually, lets wait for Jenkins to run.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        +1 LGTM, pending checkins

        Show
        rohithsharma Rohith Sharma K S added a comment - +1 LGTM, pending checkins
        Hide
        sunilg Sunil G added a comment -

        Attaching new patch.

        New command name is "changeQueue". Also improved the help message as per earlier discussion. Kindly review.

        Show
        sunilg Sunil G added a comment - Attaching new patch. New command name is "changeQueue". Also improved the help message as per earlier discussion. Kindly review.
        Hide
        yufeigu Yufei Gu added a comment -

        Yes, I agree with you.

        Show
        yufeigu Yufei Gu added a comment - Yes, I agree with you.
        Hide
        sunilg Sunil G added a comment -

        Yufei Gu, I understood your point. Definitely "movetoqueue" was clear option, i think "changeQueue/changeToQueue" is also not very bad. It may confuse, but we can make it more clearer with help message. I think I can add more information in the "help" for changeQueue to indicate this. Some thing like

        changeQueue: Moves application to a new queue. ApplicationId can be passed using 'appId' option. 
        'movetoqueue' command is deprecated, this new command 'changeQueue' performs same functionality.
        
        Show
        sunilg Sunil G added a comment - Yufei Gu , I understood your point. Definitely "movetoqueue" was clear option, i think "changeQueue/changeToQueue" is also not very bad. It may confuse, but we can make it more clearer with help message. I think I can add more information in the "help" for changeQueue to indicate this. Some thing like changeQueue: Moves application to a new queue. ApplicationId can be passed using 'appId' option. 'movetoqueue' command is deprecated, this new command 'changeQueue' performs same functionality.
        Hide
        yufeigu Yufei Gu added a comment -

        Make sense to me. Thanks to point out.
        It would be swell to be explicit about the direction, like "changetoqueue", “migratetoqueue”. It is obvious for the people who actually write the command to move app that the parameter queue should be the target queue instead of source queue, but may confuse some admins or support guys who maintain system with pre-baked scripts.

        Show
        yufeigu Yufei Gu added a comment - Make sense to me. Thanks to point out. It would be swell to be explicit about the direction, like "changetoqueue", “migratetoqueue”. It is obvious for the people who actually write the command to move app that the parameter queue should be the target queue instead of source queue, but may confuse some admins or support guys who maintain system with pre-baked scripts.
        Hide
        sunilg Sunil G added a comment -

        Thanks Yufei Gu

        Generally I understood your point. Since we already have a "movetoqueue" implemented in a different way, i think we need to deprecate the same. But we could not remove the internal code as its formally released cli. We could remove from help, and create a new cli for now.

        I think "changeQueue" may be more meaningful in that line since we cant use "movetoqueue"

        For eg:

        ./yarn application -appId application_1479894790219_0002 -changeQueue default
        

        Thoughts?

        Show
        sunilg Sunil G added a comment - Thanks Yufei Gu Generally I understood your point. Since we already have a "movetoqueue" implemented in a different way, i think we need to deprecate the same. But we could not remove the internal code as its formally released cli. We could remove from help, and create a new cli for now. I think "changeQueue" may be more meaningful in that line since we cant use "movetoqueue" For eg: ./yarn application -appId application_1479894790219_0002 -changeQueue default Thoughts?
        Hide
        yufeigu Yufei Gu added a comment -

        Thanks Sunil G for working on this.
        What about use keyword "movetoqueue" instead of "move"? e.g.

        ./yarn application -appId application_1479894790219_0002 -movetoqueue default
        

        "movetoqueue" is more explicitly about what is the following parameters in two ways, it is a queue and it is a queue that application is going to be moved to, not moved from.

        Show
        yufeigu Yufei Gu added a comment - Thanks Sunil G for working on this. What about use keyword "movetoqueue" instead of "move"? e.g. ./yarn application -appId application_1479894790219_0002 -movetoqueue default "movetoqueue" is more explicitly about what is the following parameters in two ways, it is a queue and it is a queue that application is going to be moved to, not moved from.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 50s trunk passed
        +1 compile 0m 21s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 19s the patch passed
        +1 compile 0m 18s the patch passed
        +1 javac 0m 18s the patch passed
        -0 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 4 new + 150 unchanged - 1 fixed = 154 total (was 151)
        +1 mvnsite 0m 22s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 34s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 16m 18s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        29m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6001
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843176/YARN-6001.0001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c4d651b61deb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1f14f6d
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14294/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14294/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14294/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 50s trunk passed +1 compile 0m 21s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed -0 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 4 new + 150 unchanged - 1 fixed = 154 total (was 151) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 34s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 16m 18s hadoop-yarn-client in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 29m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6001 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843176/YARN-6001.0001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c4d651b61deb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1f14f6d Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14294/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14294/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14294/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        Uploading an initial version of patch.

        cc/Rohith Sharma K S Jian He and Wangda Tan

        Show
        sunilg Sunil G added a comment - Uploading an initial version of patch. cc/ Rohith Sharma K S Jian He and Wangda Tan

          People

          • Assignee:
            sunilg Sunil G
            Reporter:
            sunilg Sunil G
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development