Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4270

Limit application resource reservation on nodes for non-node/rack specific requests

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None

      Description

      I has been noticed that for off-switch requests, the FairScheduler reserves resources on all nodes. This could lead to the entire cluster being unavailable for all other applications.

      Ideally, the reservations should be on a configurable number of nodes, default 1.

      1. YARN-4270.1.patch
        22 kB
        Arun Suresh
      2. YARN-4270.2.patch
        23 kB
        Arun Suresh
      3. YARN-4270.3.patch
        22 kB
        Arun Suresh
      4. YARN-4270.4.patch
        22 kB
        Arun Suresh
      5. YARN-4270.5.patch
        22 kB
        Arun Suresh

        Issue Links

          Activity

          Hide
          asuresh Arun Suresh added a comment -

          Attaching initial patch

          Show
          asuresh Arun Suresh added a comment - Attaching initial patch
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 51s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 50s There were no new javac warning messages.
          +1 javadoc 10m 18s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 32s The applied patch generated 3 new checkstyle issues (total was 211, now 213).
          -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 32s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 0m 22s Tests failed in hadoop-yarn-api.
          -1 yarn tests 61m 33s Tests failed in hadoop-yarn-server-resourcemanager.
              106m 23s  



          Reason Tests
          Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767236/YARN-4270.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 58590fe
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/whitespace.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9470/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9470/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 51s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 10m 18s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 32s The applied patch generated 3 new checkstyle issues (total was 211, now 213). -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 0m 22s Tests failed in hadoop-yarn-api. -1 yarn tests 61m 33s Tests failed in hadoop-yarn-server-resourcemanager.     106m 23s   Reason Tests Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767236/YARN-4270.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 58590fe checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/whitespace.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9470/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9470/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9470/console This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment -

          Updating patch :

          1. Restricted changes to only the FairScheduler
          2. Some changes to testcase
          Show
          asuresh Arun Suresh added a comment - Updating patch : Restricted changes to only the FairScheduler Some changes to testcase
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 57s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 50s There were no new javac warning messages.
          +1 javadoc 10m 24s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 49s The applied patch generated 3 new checkstyle issues (total was 128, now 128).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 61m 57s Tests failed in hadoop-yarn-server-resourcemanager.
              101m 58s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767243/YARN-4270.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 58590fe
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9471/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9471/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9471/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9471/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 10m 24s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 49s The applied patch generated 3 new checkstyle issues (total was 128, now 128). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 61m 57s Tests failed in hadoop-yarn-server-resourcemanager.     101m 58s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767243/YARN-4270.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 58590fe checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9471/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9471/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9471/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9471/console This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment -

          Previous test case failure was spurious.

          Updating Patch anyway:

          1. Added one more testcase to verify behavior for rack local reservations
          2. Fixed checkstyle warnings

          Karthik Kambatla, do you want to give it a quick rev ?

          Show
          asuresh Arun Suresh added a comment - Previous test case failure was spurious. Updating Patch anyway: Added one more testcase to verify behavior for rack local reservations Fixed checkstyle warnings Karthik Kambatla , do you want to give it a quick rev ?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 45s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 15s There were no new javac warning messages.
          +1 javadoc 10m 37s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 52s The applied patch generated 1 new checkstyle issues (total was 128, now 129).
          +1 whitespace 0m 3s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          +1 findbugs 1m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 62m 15s Tests passed in hadoop-yarn-server-resourcemanager.
              103m 51s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767259/YARN-4270.3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0ab3f9d
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9473/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9473/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9473/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9473/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 45s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 15s There were no new javac warning messages. +1 javadoc 10m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 52s The applied patch generated 1 new checkstyle issues (total was 128, now 129). +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 1m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 62m 15s Tests passed in hadoop-yarn-server-resourcemanager.     103m 51s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767259/YARN-4270.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0ab3f9d checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9473/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9473/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9473/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9473/console This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment -

          Addressing checkstyle warning..

          Show
          asuresh Arun Suresh added a comment - Addressing checkstyle warning..
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 33s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 11s There were no new javac warning messages.
          +1 javadoc 10m 47s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 50s There were no new checkstyle issues.
          +1 whitespace 0m 3s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 63m 16s Tests passed in hadoop-yarn-server-resourcemanager.
              104m 48s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767262/YARN-4270.4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0ab3f9d
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9474/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9474/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9474/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 11s There were no new javac warning messages. +1 javadoc 10m 47s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 50s There were no new checkstyle issues. +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 63m 16s Tests passed in hadoop-yarn-server-resourcemanager.     104m 48s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767262/YARN-4270.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0ab3f9d hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9474/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9474/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9474/console This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for reporting and working on this, Arun. The approach looks good to me. Few minor comments on the patch:

          1. Can we call the config and variables reservableNodes? Calling it appReservationThreshold increases the ambiguity with SLA-reservations.
          2. In FairScheduler#removeNode, can we log an error in the else block. i.e., when a rack is not found in the app or has a node-count <=0?
          3. FSAppAttempt: can we add a comment to describe what the map holds? i.e., what String corresponds to?
              private Map<String, Set<String>> reservations = new HashMap<>();
            
          4. Should we use a smaller value for the default? Consider a cluster with 2k nodes: with this default, we end up reserving on 1000 nodes. How about using 0.05?
          5. I believe we should be reserving nodes for an app only after the locality of the corresponding RR is fully relaxed. That is likely for another JIRA.
          6. Nit: Spurious changes in FairSchedulerConfiguration
          Show
          kasha Karthik Kambatla added a comment - Thanks for reporting and working on this, Arun. The approach looks good to me. Few minor comments on the patch: Can we call the config and variables reservableNodes ? Calling it appReservationThreshold increases the ambiguity with SLA-reservations. In FairScheduler#removeNode , can we log an error in the else block. i.e., when a rack is not found in the app or has a node-count <=0? FSAppAttempt: can we add a comment to describe what the map holds? i.e., what String corresponds to? private Map< String , Set< String >> reservations = new HashMap<>(); Should we use a smaller value for the default? Consider a cluster with 2k nodes: with this default, we end up reserving on 1000 nodes. How about using 0.05? I believe we should be reserving nodes for an app only after the locality of the corresponding RR is fully relaxed. That is likely for another JIRA. Nit: Spurious changes in FairSchedulerConfiguration
          Hide
          kasha Karthik Kambatla added a comment -

          Added comment too soon. Couple of comments:

          1. FSAppAttempt#reservationExceedsThreshold: Instead of doing a ratio, would it be better to check existingReservations > Math.ceil(reservableNodes() * totalAvailableNodes)? We avoid a division and the check to see if there is at least one reservation is not necessary.
          2. Nice tests.
          Show
          kasha Karthik Kambatla added a comment - Added comment too soon. Couple of comments: FSAppAttempt#reservationExceedsThreshold: Instead of doing a ratio, would it be better to check existingReservations > Math.ceil(reservableNodes() * totalAvailableNodes) ? We avoid a division and the check to see if there is at least one reservation is not necessary. Nice tests.
          Hide
          asuresh Arun Suresh added a comment -

          Thanks for the review Karthik Kambatla !!

          Updating patch with your suggestions. Will file a follow up for delaying reservations until locality relaxation has maxed out.

          Show
          asuresh Arun Suresh added a comment - Thanks for the review Karthik Kambatla !! Updating patch with your suggestions. Will file a follow up for delaying reservations until locality relaxation has maxed out.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 20m 8s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 59s There were no new javac warning messages.
          +1 javadoc 11m 35s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 55s There were no new checkstyle issues.
          +1 whitespace 0m 4s The patch has no lines that end in whitespace.
          +1 install 1m 41s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 1m 44s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 58m 40s Tests passed in hadoop-yarn-server-resourcemanager.
              104m 54s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767467/YARN-4270.5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6144e01
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9484/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9484/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9484/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 8s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 59s There were no new javac warning messages. +1 javadoc 11m 35s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 55s There were no new checkstyle issues. +1 whitespace 0m 4s The patch has no lines that end in whitespace. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 1m 44s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 58m 40s Tests passed in hadoop-yarn-server-resourcemanager.     104m 54s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767467/YARN-4270.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6144e01 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9484/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9484/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9484/console This message was automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          +1

          Show
          kasha Karthik Kambatla added a comment - +1
          Hide
          asuresh Arun Suresh added a comment -

          Committed to trunk and branch-2. Thanks again for the review Karthik Kambatla

          Show
          asuresh Arun Suresh added a comment - Committed to trunk and branch-2. Thanks again for the review Karthik Kambatla
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8668 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8668/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8668 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8668/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1291/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1291/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #555 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/555/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #555 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/555/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #571 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/571/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #571 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/571/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2504 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2504/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2504 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2504/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2454 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2454/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2454 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2454/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #517 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/517/)
          YARN-4270. Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #517 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/517/ ) YARN-4270 . Limit application resource reservation on nodes for (arun suresh: rev 7e2837f830382835838c82398db6fc9823d612a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java

            People

            • Assignee:
              asuresh Arun Suresh
              Reporter:
              asuresh Arun Suresh
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development