Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: scheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Implement a FairOrderingPolicy which prefers to allocate to SchedulerProcesses with least current usage, very similar to the FairScheduler's FairSharePolicy.

      The Policy will offer allocations to applications in a queue in order of least resources used, and preempt applications in reverse order (from most resources used). This will include conditional support for sizeBasedWeight style adjustment

      Optionally, based on a conditional configuration to enable sizeBasedWeight (default false), an adjustment to boost larger applications (to offset the natural preference for smaller applications) will adjust the resource usage value based on demand, dividing it by the below value:

      Math.log1p(app memory demand) / Math.log(2);

      In cases where the above is indeterminate (two applications are equal after this comparison), behavior falls back to comparison based on the application id, which is generally lexically FIFO for that comparison

      1. YARN-3319.13.patch
        9 kB
        Craig Welch
      2. YARN-3319.14.patch
        7 kB
        Craig Welch
      3. YARN-3319.17.patch
        10 kB
        Craig Welch
      4. YARN-3319.35.patch
        18 kB
        Craig Welch
      5. YARN-3319.39.patch
        18 kB
        Craig Welch
      6. YARN-3319.45.patch
        10 kB
        Craig Welch
      7. YARN-3319.47.patch
        10 kB
        Craig Welch
      8. YARN-3319.53.patch
        10 kB
        Craig Welch
      9. YARN-3319.58.patch
        13 kB
        Craig Welch
      10. YARN-3319.70.patch
        20 kB
        Craig Welch
      11. YARN-3319.71.patch
        22 kB
        Craig Welch
      12. YARN-3319.72.patch
        22 kB
        Craig Welch
      13. YARN-3319.73.patch
        22 kB
        Craig Welch
      14. YARN-3319.74.patch
        23 kB
        Craig Welch
      15. YARN-3319.75.patch
        23 kB
        Craig Welch

        Issue Links

          Activity

          Hide
          cwelch Craig Welch added a comment -

          Initially this will be implemented for SchedulerApplicationAttempts in the CapacityScheduler LeafQueue (similar to the FIFO implementation in YARN-3318). The expectation is that this will be implement the SchedulerComparator interface and will be used as a comparator within the SchedulerComparatorPolicy implementation to achieve the intended behavior.

          Show
          cwelch Craig Welch added a comment - Initially this will be implemented for SchedulerApplicationAttempts in the CapacityScheduler LeafQueue (similar to the FIFO implementation in YARN-3318 ). The expectation is that this will be implement the SchedulerComparator interface and will be used as a comparator within the SchedulerComparatorPolicy implementation to achieve the intended behavior.
          Hide
          cwelch Craig Welch added a comment -

          Attaching initial/incomplete patch, it depends on the YARN-3318 patch of the same index - it is just the additional logic specific to Fairness. Major TODO, sizeBasedWeight.

          Show
          cwelch Craig Welch added a comment - Attaching initial/incomplete patch, it depends on the YARN-3318 patch of the same index - it is just the additional logic specific to Fairness. Major TODO, sizeBasedWeight.
          Hide
          cwelch Craig Welch added a comment -

          Same as .13, except it should be possible to apply this patch after applying YARN-3318 's .14 patch

          Show
          cwelch Craig Welch added a comment - Same as .13, except it should be possible to apply this patch after applying YARN-3318 's .14 patch
          Hide
          cwelch Craig Welch added a comment -

          With support for configuration via the scheduler's config file

          Show
          cwelch Craig Welch added a comment - With support for configuration via the scheduler's config file
          Hide
          cwelch Craig Welch added a comment -

          Apply after applying YARN-3318.35.patch

          Show
          cwelch Craig Welch added a comment - Apply after applying YARN-3318 .35.patch
          Hide
          cwelch Craig Welch added a comment -

          Apply after applying YARN-3318 and YARN-3463

          Show
          cwelch Craig Welch added a comment - Apply after applying YARN-3318 and YARN-3463
          Hide
          cwelch Craig Welch added a comment -

          Up to date with same patch index on YARN-3463, still one todo, which is to add the "fair" -> FairPolicyClassName translation in the scheduler configuration. Will finalize after YARN-3463 commit

          Show
          cwelch Craig Welch added a comment - Up to date with same patch index on YARN-3463 , still one todo, which is to add the "fair" -> FairPolicyClassName translation in the scheduler configuration. Will finalize after YARN-3463 commit
          Hide
          cwelch Craig Welch added a comment -

          with "fair" config support, sizeBasedWeight config support

          Show
          cwelch Craig Welch added a comment - with "fair" config support, sizeBasedWeight config support
          Hide
          cwelch Craig Welch added a comment -

          Fix a findbugs

          Show
          cwelch Craig Welch added a comment - Fix a findbugs
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726734/YARN-3319.72.patch
          against trunk revision 44872b7.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7413//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7413//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7413//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726734/YARN-3319.72.patch against trunk revision 44872b7. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7413//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7413//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7413//console This message is automatically generated.
          Hide
          cwelch Craig Welch added a comment -

          another findbugs

          Show
          cwelch Craig Welch added a comment - another findbugs
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726913/YARN-3319.73.patch
          against trunk revision 8ddbb8d.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7426//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7426//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7426//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726913/YARN-3319.73.patch against trunk revision 8ddbb8d. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7426//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7426//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7426//console This message is automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Some minor comments about configuration part:
          1) getting config mapping should be a generic method, you can use Configuration.iterator() to get all properties of a policy.
          2) Policy will handle properties such as "size-based-weight" in its own configure(..)
          3) Do you think is it better to make property in <queue-name>.ordering-policy.<policy-name>.<property-key>? In your example, "sizeBasedWeight" more like a property of fair policy instead of others. I perfer add a <policy-name>
          4) "sizeBasedWeight" to "size-based-weight-enabled" (or enable-size-based-weight) to make it consistent with other configurations of CS. Add the enable to indicate it's a boolean instead of float.
          5) Make the String="size-based-weight" to a final static variable like other configuration keys. Same of "fifo" and "fifo". Test should use these final static variables as well

          FairOrderingPolicy:
          1) Mark get/setSizeBasedWeight to VisibleForTesting?
          2) getMagnitude should be private
          3) LOG is not used by anybody, could be removed.

          Findbugs warning?

          Show
          leftnoteasy Wangda Tan added a comment - Some minor comments about configuration part: 1) getting config mapping should be a generic method, you can use Configuration.iterator() to get all properties of a policy. 2) Policy will handle properties such as "size-based-weight" in its own configure(..) 3) Do you think is it better to make property in <queue-name>.ordering-policy.<policy-name>.<property-key>? In your example, "sizeBasedWeight" more like a property of fair policy instead of others. I perfer add a <policy-name> 4) "sizeBasedWeight" to "size-based-weight-enabled" (or enable-size-based-weight) to make it consistent with other configurations of CS. Add the enable to indicate it's a boolean instead of float. 5) Make the String="size-based-weight" to a final static variable like other configuration keys. Same of "fifo" and "fifo". Test should use these final static variables as well FairOrderingPolicy: 1) Mark get/setSizeBasedWeight to VisibleForTesting? 2) getMagnitude should be private 3) LOG is not used by anybody, could be removed. Findbugs warning?
          Hide
          cwelch Craig Welch added a comment -

          Some minor comments about configuration part

          by index:
          1) done
          2) done
          3) done - see below

          Do you think is it better to make property in <queue-name>.ordering-policy.<policy-name>.<property-key>?...

          Now that there is not proper composition only one policy can be active at a time and it shouldn't be necessary to namespace config items this way. At the same time, I could see us getting back to proper composition at some point, where this would be helpful. I've implemented it as a prefix convention in the policy instead of constraining the contents of the map in the capacity scheduler configuration. This is because we still support passing a class name as the policy type, which would make the configurations for "class name based" items unwieldy. It would also allow us to have shared configuration items between policies if we do end up with proper composition again. The end result of the configuration was as you suggested
          4) done
          5) done

          FairOrderingPolicy:

          all 3 done

          Findbugs warning?

          Failed to stage change, so it didn't make it into patch, should be there now.

          Show
          cwelch Craig Welch added a comment - Some minor comments about configuration part by index: 1) done 2) done 3) done - see below Do you think is it better to make property in <queue-name>.ordering-policy.<policy-name>.<property-key>?... Now that there is not proper composition only one policy can be active at a time and it shouldn't be necessary to namespace config items this way. At the same time, I could see us getting back to proper composition at some point, where this would be helpful. I've implemented it as a prefix convention in the policy instead of constraining the contents of the map in the capacity scheduler configuration. This is because we still support passing a class name as the policy type, which would make the configurations for "class name based" items unwieldy. It would also allow us to have shared configuration items between policies if we do end up with proper composition again. The end result of the configuration was as you suggested 4) done 5) done FairOrderingPolicy: all 3 done Findbugs warning? Failed to stage change, so it didn't make it into patch, should be there now.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for update, Craig Welch. Patch LGTM, +1.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for update, Craig Welch . Patch LGTM, +1.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 59s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 whitespace 0m 0s The patch has 99 line(s) that end in whitespace.
          +1 javac 7m 45s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 5m 25s The applied patch generated 3 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 17s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          -1 yarn tests 50m 50s Tests failed in hadoop-yarn-server-resourcemanager.
              92m 21s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestRM
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12727310/YARN-3319.74.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 12f4df0
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/whitespace.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7454/testReport/
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7454//console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 59s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 whitespace 0m 0s The patch has 99 line(s) that end in whitespace. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 25s The applied patch generated 3 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 17s The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 yarn tests 50m 50s Tests failed in hadoop-yarn-server-resourcemanager.     92m 21s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727310/YARN-3319.74.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 12f4df0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/whitespace.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7454/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7454/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7454//console This message was automatically generated.
          Hide
          cwelch Craig Welch added a comment -

          The failed tests pass on my box with the patch, unrelated. The checkstyle is referring to ResourceLimits, which the patch doesn't change... poking around in the build artifacts there are some exceptions in some of the checkstyle stuff, I'm not sure it's actually working correctly

          Show
          cwelch Craig Welch added a comment - The failed tests pass on my box with the patch, unrelated. The checkstyle is referring to ResourceLimits, which the patch doesn't change... poking around in the build artifacts there are some exceptions in some of the checkstyle stuff, I'm not sure it's actually working correctly
          Hide
          leftnoteasy Wangda Tan added a comment -

          Retriggerred Jenkins to see if problem still exists.

          Show
          leftnoteasy Wangda Tan added a comment - Retriggerred Jenkins to see if problem still exists.
          Hide
          cwelch Craig Welch added a comment -

          trying something wrt checkstyle

          Show
          cwelch Craig Welch added a comment - trying something wrt checkstyle
          Hide
          leftnoteasy Wangda Tan added a comment -

          Fixed whitespace issues locally, committed to trunk/branch-2, thanks Craig Welch!

          Show
          leftnoteasy Wangda Tan added a comment - Fixed whitespace issues locally, committed to trunk/branch-2, thanks Craig Welch !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7648 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7648/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7648 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7648/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Is FairOrderingPolicy.ENABLE_SIZE_BASED_WEIGHT supposed to be admin visible? If so, we need a better, fully qualified name..

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Is FairOrderingPolicy.ENABLE_SIZE_BASED_WEIGHT supposed to be admin visible? If so, we need a better, fully qualified name..
          Hide
          cwelch Craig Welch added a comment -

          Yes, it's configured in the capacity scheduler configuration with something like this:

          <property>
              <name>(yarn-queue-prefix).ordering-policy.fair.enable-size-based-weight</name>
              <value>true</value>
            </property>
          
          Show
          cwelch Craig Welch added a comment - Yes, it's configured in the capacity scheduler configuration with something like this: <property> <name>(yarn-queue-prefix).ordering-policy.fair.enable-size-based-weight</name> <value> true </value> </property>
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/)
          YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/ ) YARN-3319 . Implement a FairOrderingPolicy. (Craig Welch via wangda) (wangda: rev 395205444e8a9ae6fc86f0a441e98486a775511a) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/CompoundComparator.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/FairOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java

            People

            • Assignee:
              cwelch Craig Welch
              Reporter:
              cwelch Craig Welch
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development