Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      document changes only.
    • Target Version/s:

      Description

      I noticed the following issues with the capacity scheduler doc: ./hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm

      • In overview section, 3rd paragraph, sentence "There is an added benefit that an organization can access any excess capacity no being used by others". No should be not.
      • in overview section, 4th paragraph. dispropotionate misspelled
      • in features section, under multitenancy - monopolizing is misspelled.
      • in features section, under operability - it doesn't say if you can delete queues at runtime? I see there is a note at the end but perhaps that can be added into the other sections to since its easy to miss that Note at the very end.
      • in features section - hierarchy and Hierarchical mispelled.
      • under configuration section the class to turn on to use capacity scheduler should be: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
      • section on setting up queues, 4th sentence - hierarchy misspelled as heirarcy and heirarchy.
      • I think specifying how a user has to specify the queue when running a job/app would be useful information. Especially with the new hierarchical queues. Does the user have to specify the entire path like a.b.c or can they just specify c.
      • under "Running and Pending Application Limits" section, property "yarn.scheduler.capacity.maximum-applications", they are referred to them as jobs, I believe that should be applications.
      • misspelled concurrently as concurently in same section of maximum-applications.
      • I think it should specify the defaults (if any) for the config vars. Also what format are they specified in - int, float,etc?
      • might be nice to say it doesn't support preemption.
      • under admin options yarn.scheduler.capacity.<queue-path>.state - queues misspelled as queueus
      • under changing queue configuration it should have "yarn" in front of the "rmadmin -refreshQueues". Similarly a few lines down at "$YARN_HOME/bin/rmadmin -refreshQueues"

        Issue Links

          Activity

          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12501980/MAPREDUCE-3325.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1241//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1241//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501980/MAPREDUCE-3325.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1241//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1241//console This message is automatically generated.
          Hide
          Thomas Graves added a comment -

          documentation changes only - tests don't apply.

          Show
          Thomas Graves added a comment - documentation changes only - tests don't apply.
          Hide
          Mahadev konar added a comment -

          +1 looks good.

          Show
          Mahadev konar added a comment - +1 looks good.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1343 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1343/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1343 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1343/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Mahadev konar added a comment -

          Just committed this. Thanks Thomas.

          Show
          Mahadev konar added a comment - Just committed this. Thanks Thomas.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1269 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1269/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1269 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1269/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #168 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/168/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #168 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/168/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #169 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/169/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #169 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/169/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1291/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1291/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #180 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/180/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #180 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/180/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #864 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/864/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #864 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/864/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #77 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/77/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #77 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/77/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #94 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/94/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #94 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/94/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) - Merging r1201925 from trunk mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201927 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #898 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/898/)
          MAPREDUCE-3325. Improvements to CapacityScheduler doc. (Thomas Graves via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #898 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/898/ ) MAPREDUCE-3325 . Improvements to CapacityScheduler doc. (Thomas Graves via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1201925 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm

            People

            • Assignee:
              Thomas Graves
              Reporter:
              Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development