Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3446

FairScheduler headroom calculation should exclude nodes in the blacklist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FairScheduler HeadRoom calculation should exclude nodes in the blacklist.
      MRAppMaster does not preempt the reducers because for Reducer preemption calculation, headRoom is considering blacklisted nodes. This makes jobs to hang forever(ResourceManager does not assign any new containers on blacklisted nodes but availableResource AM get from RM includes blacklisted nodes available resource).
      This issue is similar as YARN-1680 which is for Capacity Scheduler.

      1. YARN-3446.000.patch
        11 kB
        zhihai xu
      2. YARN-3446.001.patch
        10 kB
        zhihai xu
      3. YARN-3446.002.patch
        11 kB
        zhihai xu
      4. YARN-3446.003.patch
        17 kB
        zhihai xu
      5. YARN-3446.004.patch
        16 kB
        zhihai xu
      6. YARN-3446.005.patch
        16 kB
        zhihai xu
      7. YARN-3446-branch-2.8.txt
        18 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12725478/YARN-3446.000.patch
          against trunk revision fddd552.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7343//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7343//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12725478/YARN-3446.000.patch against trunk revision fddd552. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7343//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7343//console This message is automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          It's not entirely clear this is the solution we want. Can we please hold this patch off while we figure out the right solution at YARN-1680?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - It's not entirely clear this is the solution we want. Can we please hold this patch off while we figure out the right solution at YARN-1680 ?
          Hide
          kasha Karthik Kambatla added a comment -

          Discussing the approach on YARN-1680. Let us finalize the approach there quickly, and make progress here.

          Show
          kasha Karthik Kambatla added a comment - Discussing the approach on YARN-1680 . Let us finalize the approach there quickly, and make progress here.
          Hide
          kasha Karthik Kambatla added a comment -

          zhihai xu - the patch doesn't apply any more. Could you please rebase it onto trunk?

          Show
          kasha Karthik Kambatla added a comment - zhihai xu - the patch doesn't apply any more. Could you please rebase it onto trunk?
          Hide
          zxu zhihai xu added a comment -

          Thanks Karthik Kambatla for the reminder! I just uploaded a new patch YARN-3446.002.patch based on the latest code at trunk. Please review it.

          Show
          zxu zhihai xu added a comment - Thanks Karthik Kambatla for the reminder! I just uploaded a new patch YARN-3446 .002.patch based on the latest code at trunk. Please review it.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for rebasing the patch, zhihai xu. Comments:

          FSAppAttempt:

          1. How about using a helper method subtractResourcesOnBlacklistedNodes instead of adding all the logic to getHeadroom itself?
          2. Is the optimization to get the blacklist only when it has changed necessary? Looks like we optimize the fetch, but not the iteration on it. I think we should either go all the way and optimize iterating on the blacklist nodes as well only when the blacklist has changed, or leave out the optimization until we see a need for it.
          3. To get the blacklist, can't we just use AppSchedulingInfo#getBlacklist (needs synchronization) or AppSchedulingInfo#getBlacklistCopy? Do we need the methods in the scheduler?

          If we make these changes, we might not need all the changes in rest of the files.

          Show
          kasha Karthik Kambatla added a comment - Thanks for rebasing the patch, zhihai xu . Comments: FSAppAttempt: How about using a helper method subtractResourcesOnBlacklistedNodes instead of adding all the logic to getHeadroom itself? Is the optimization to get the blacklist only when it has changed necessary? Looks like we optimize the fetch, but not the iteration on it. I think we should either go all the way and optimize iterating on the blacklist nodes as well only when the blacklist has changed, or leave out the optimization until we see a need for it. To get the blacklist, can't we just use AppSchedulingInfo#getBlacklist (needs synchronization) or AppSchedulingInfo#getBlacklistCopy ? Do we need the methods in the scheduler? If we make these changes, we might not need all the changes in rest of the files.
          Hide
          zxu zhihai xu added a comment -

          Hi Karthik Kambatla, Thanks for the review! I attached a new patch YARN-3446.003.patch, which addressed your first comment. I also added more test cases to verify getHeadroom with blacklisted nodes remove and addition.
          About your second comment: IMHO, if we didn't do the optimization, that will be a very big overhead for a large cluster. For example, we have 2000 AM running on 5000 nodes cluster, For each AM, we need go through 5000 nodes list to find the blacklisted SchedulerNode in the heartbeat. With 2000 AM, it will loop 10,000,000 times. Normally number of blacklisted nodes should be very small for each application. So iterating on the blacklisted nodes may not be a performance issue. Also AM won't change blacklisted nodes frequently.
          About your third comment, it is because currently SchedulerNode are stored in AbstractYarnScheduler#nodes with key NodeId. But AppSchedulingInfo stores the blacklisted nodes using String Node Name or Rack Name. I can't find an easy way to translate Node Name and Rack Name to NodeId. So it looks like we need iterate through AbstractYarnScheduler#nodes to find the blacklisted SchedulerNode if we use AppSchedulingInfo#getBlacklist. That means for a 5000 nodes cluster, we need loop 5000 times, a big overhead. AbstractYarnScheduler#nodes are defined at the following code:

            protected Map<NodeId, N> nodes = new ConcurrentHashMap<NodeId, N>();
          
          Show
          zxu zhihai xu added a comment - Hi Karthik Kambatla , Thanks for the review! I attached a new patch YARN-3446 .003.patch, which addressed your first comment. I also added more test cases to verify getHeadroom with blacklisted nodes remove and addition. About your second comment: IMHO, if we didn't do the optimization, that will be a very big overhead for a large cluster. For example, we have 2000 AM running on 5000 nodes cluster, For each AM, we need go through 5000 nodes list to find the blacklisted SchedulerNode in the heartbeat. With 2000 AM, it will loop 10,000,000 times. Normally number of blacklisted nodes should be very small for each application. So iterating on the blacklisted nodes may not be a performance issue. Also AM won't change blacklisted nodes frequently. About your third comment, it is because currently SchedulerNode are stored in AbstractYarnScheduler#nodes with key NodeId . But AppSchedulingInfo stores the blacklisted nodes using String Node Name or Rack Name. I can't find an easy way to translate Node Name and Rack Name to NodeId . So it looks like we need iterate through AbstractYarnScheduler#nodes to find the blacklisted SchedulerNode if we use AppSchedulingInfo#getBlacklist . That means for a 5000 nodes cluster, we need loop 5000 times, a big overhead. AbstractYarnScheduler#nodes are defined at the following code: protected Map<NodeId, N> nodes = new ConcurrentHashMap<NodeId, N>();
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 33s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 47s There were no new javac warning messages.
          +1 javadoc 10m 3s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 50s The applied patch generated 4 new checkstyle issues (total was 89, now 92).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 58m 31s Tests passed in hadoop-yarn-server-resourcemanager.
              97m 44s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12761350/YARN-3446.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 3a9c707
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9224/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9224/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9224/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9224/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 50s The applied patch generated 4 new checkstyle issues (total was 89, now 92). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 58m 31s Tests passed in hadoop-yarn-server-resourcemanager.     97m 44s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761350/YARN-3446.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 3a9c707 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9224/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9224/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9224/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9224/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 42s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 51s There were no new javac warning messages.
          +1 javadoc 10m 4s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 51s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 59m 10s Tests passed in hadoop-yarn-server-resourcemanager.
              98m 35s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12761356/YARN-3446.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 3a9c707
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9225/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9225/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9225/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 42s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 51s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 59m 10s Tests passed in hadoop-yarn-server-resourcemanager.     98m 35s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761356/YARN-3446.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 3a9c707 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9225/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9225/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9225/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 36s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 9m 7s There were no new javac warning messages.
          +1 javadoc 11m 48s There were no new javadoc warning messages.
          -1 release audit 0m 16s The applied patch generated 1 release audit warnings.
          +1 checkstyle 0m 56s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
          +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 57m 47s Tests failed in hadoop-yarn-server-resourcemanager.
              103m 34s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765274/YARN-3446.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6d5713a
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9367/artifact/patchprocess/patchReleaseAuditProblems.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9367/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9367/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9367/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 9m 7s There were no new javac warning messages. +1 javadoc 11m 48s There were no new javadoc warning messages. -1 release audit 0m 16s The applied patch generated 1 release audit warnings. +1 checkstyle 0m 56s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 1m 41s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 57m 47s Tests failed in hadoop-yarn-server-resourcemanager.     103m 34s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765274/YARN-3446.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6d5713a Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9367/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9367/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9367/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9367/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 57s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 8m 20s There were no new javac warning messages.
          +1 javadoc 10m 26s There were no new javadoc warning messages.
          -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
          +1 checkstyle 0m 48s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 61m 11s Tests passed in hadoop-yarn-server-resourcemanager.
              102m 41s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766024/YARN-3446.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / db93047
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9403/artifact/patchprocess/patchReleaseAuditProblems.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9403/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9403/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9403/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 20s There were no new javac warning messages. +1 javadoc 10m 26s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. +1 checkstyle 0m 48s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 61m 11s Tests passed in hadoop-yarn-server-resourcemanager.     102m 41s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766024/YARN-3446.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / db93047 Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9403/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9403/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9403/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9403/console This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Sorry for not looking at this since the last update. Looks like the patch doesn't apply anymore.

          zhihai xu - mind updating the patch? I ll take a look more promptly this time.

          Show
          kasha Karthik Kambatla added a comment - Sorry for not looking at this since the last update. Looks like the patch doesn't apply anymore. zhihai xu - mind updating the patch? I ll take a look more promptly this time.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 4s YARN-3446 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12766024/YARN-3446.003.patch
          JIRA Issue YARN-3446
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10147/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 4s YARN-3446 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12766024/YARN-3446.003.patch JIRA Issue YARN-3446 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10147/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          thanks for the review! Just updated the patch at YARN-3446.004.patch.

          Show
          zxu zhihai xu added a comment - thanks for the review! Just updated the patch at YARN-3446 .004.patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 2s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_66
          +1 compile 0m 32s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 39s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 18s trunk passed
          +1 javadoc 0m 24s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 33s the patch passed
          +1 compile 0m 28s the patch passed with JDK v1.8.0_66
          +1 javac 0m 28s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.7.0_91
          +1 javac 0m 31s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 37s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 findbugs 1m 22s the patch passed
          +1 javadoc 0m 21s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 27s the patch passed with JDK v1.7.0_91
          -1 unit 62m 40s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 61m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          142m 46s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780487/YARN-3446.004.patch
          JIRA Issue YARN-3446
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f99baf82d8a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6da6d87
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10151/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10151/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 2s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_66 +1 compile 0m 32s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 18s trunk passed +1 javadoc 0m 24s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 33s the patch passed +1 compile 0m 28s the patch passed with JDK v1.8.0_66 +1 javac 0m 28s the patch passed +1 compile 0m 31s the patch passed with JDK v1.7.0_91 +1 javac 0m 31s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 37s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 1m 22s the patch passed +1 javadoc 0m 21s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 27s the patch passed with JDK v1.7.0_91 -1 unit 62m 40s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 61m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 142m 46s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780487/YARN-3446.004.patch JIRA Issue YARN-3446 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f99baf82d8a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6da6d87 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10151/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10151/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10151/console This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Patch looks good, but for one minor comment: can we rename AbstractYarnScheduler#getBlackListNodeIds to addBlacklisedNodeIdsToList to capture the behavior here of adding the nodeIds to the list that is passed. Also, given the method is used by all schedulers, we might want to add a javadoc briefly explaining what it does.

          Show
          kasha Karthik Kambatla added a comment - Patch looks good, but for one minor comment: can we rename AbstractYarnScheduler#getBlackListNodeIds to addBlacklisedNodeIdsToList to capture the behavior here of adding the nodeIds to the list that is passed. Also, given the method is used by all schedulers, we might want to add a javadoc briefly explaining what it does.
          Hide
          zxu zhihai xu added a comment -

          Thanks for the review Karthik Kambatla! That is a good suggestion. I attached a new patch YARN-3446.005.patch, which addressed your comments. Please review it.

          Show
          zxu zhihai xu added a comment - Thanks for the review Karthik Kambatla ! That is a good suggestion. I attached a new patch YARN-3446 .005.patch, which addressed your comments. Please review it.
          Hide
          kasha Karthik Kambatla added a comment -

          +1, pending Jenkins. Will commit it if Jenkins says okay.

          Show
          kasha Karthik Kambatla added a comment - +1, pending Jenkins. Will commit it if Jenkins says okay.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 23s trunk passed
          +1 compile 0m 28s trunk passed with JDK v1.8.0_66
          +1 compile 0m 30s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 36s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 10s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.8.0_66
          +1 javac 0m 24s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_91
          +1 javac 0m 27s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 17s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_91
          -1 unit 59m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 61m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          137m 59s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782207/YARN-3446.005.patch
          JIRA Issue YARN-3446
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0657c4d900da 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 62b0d61
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10277/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10277/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 23s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_66 +1 compile 0m 30s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 36s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 10s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 31s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_66 +1 javac 0m 24s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_91 +1 javac 0m 27s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_91 -1 unit 59m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 61m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 137m 59s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782207/YARN-3446.005.patch JIRA Issue YARN-3446 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0657c4d900da 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 62b0d61 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10277/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10277/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10277/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          The test failures for TestClientRMTokens and TestAMAuthorizatio are not related to the patch. Both tests are passed in my local build.

          Show
          zxu zhihai xu added a comment - The test failures for TestClientRMTokens and TestAMAuthorizatio are not related to the patch. Both tests are passed in my local build.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9112 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9112/)
          YARN-3446. FairScheduler headroom calculation should exclude nodes in (kasha: rev 9d04f26d4c42170ee3dab2f6fb09a94bbf72fc65)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFSAppAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAppSchedulingInfo.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9112 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9112/ ) YARN-3446 . FairScheduler headroom calculation should exclude nodes in (kasha: rev 9d04f26d4c42170ee3dab2f6fb09a94bbf72fc65) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppSchedulingInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAppSchedulingInfo.java
          Hide
          kasha Karthik Kambatla added a comment -

          Just committed this to trunk and branch-2. Thanks zhihai xu for working through the tardy reviews.

          Show
          kasha Karthik Kambatla added a comment - Just committed this to trunk and branch-2. Thanks zhihai xu for working through the tardy reviews.
          Hide
          zxu zhihai xu added a comment -

          Karthik Kambatla, thanks for the review and committing the patch!

          Show
          zxu zhihai xu added a comment - Karthik Kambatla , thanks for the review and committing the patch!
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Uploading a patch that I just pushed in for branch-2.8. This patch is needed by YARN-4837-> YARN-4307 on branch-2.8.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Uploading a patch that I just pushed in for branch-2.8. This patch is needed by YARN-4837 -> YARN-4307 on branch-2.8.

            People

            • Assignee:
              zxu zhihai xu
              Reporter:
              zxu zhihai xu
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development