Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      After YARN-2003, adding support to change priority of an application after submission. This ticket will handle the server side implementation for same.
      A new RMAppEvent will be created to handle this, and will be common for all schedulers.

      1. 0006-YARN-3887.patch
        21 kB
        Sunil G
      2. 0005-YARN-3887.patch
        23 kB
        Sunil G
      3. 0004-YARN-3887.patch
        23 kB
        Sunil G
      4. 0003-YARN-3887.patch
        23 kB
        Sunil G
      5. 0002-YARN-3887.patch
        14 kB
        Sunil G
      6. 0001-YARN-3887.patch
        14 kB
        Sunil G

        Issue Links

          Activity

          Hide
          sunilg Sunil G added a comment -

          Uploading initial patch.

          Show
          sunilg Sunil G added a comment - Uploading initial patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 16m 58s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 49s There were no new javac warning messages.
          +1 javadoc 9m 57s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 22s There were no new checkstyle issues.
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 51m 0s Tests failed in hadoop-yarn-server-resourcemanager.
              90m 1s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            org.apache.hadoop.yarn.server.resourcemanager.TestRM
            org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12746548/0001-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4025326
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8612/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8612/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8612/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 16m 58s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 9m 57s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 22s There were no new checkstyle issues. +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 51m 0s Tests failed in hadoop-yarn-server-resourcemanager.     90m 1s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746548/0001-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4025326 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8612/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8612/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8612/console This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          thanks Sunil ! some comments on the patch:

          • Do you plan to do client side changes as part of this jira ?
          • RMAppUpdatePriorityEvent - RMApp may receive this event at many other states other than RUNNING state. In that case, the state-machine will throw InvalidEventException. I think we do not need to send event to RMApp, all it does is just get the application submission context and set the priority. This can be done at clientRMService. Similarly, the event to state-store can be sent directly from clientRMService.
          • CapacityScheuduler#updateApplicationPriority does not need to be synchronized ?
          Show
          jianhe Jian He added a comment - thanks Sunil ! some comments on the patch: Do you plan to do client side changes as part of this jira ? RMAppUpdatePriorityEvent - RMApp may receive this event at many other states other than RUNNING state. In that case, the state-machine will throw InvalidEventException. I think we do not need to send event to RMApp, all it does is just get the application submission context and set the priority. This can be done at clientRMService. Similarly, the event to state-store can be sent directly from clientRMService. CapacityScheuduler#updateApplicationPriority does not need to be synchronized ?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Hi Jian He,

          Do you plan to do client side changes as part of this jira ?

          YARN-3250 is planning to do changes for admin and user CLI i.e ApplicationClientProtocol. This jira is intended for only scheduler side chagnes support for API's. In YARN-3250, will be using these exposed API's and implementing it.
          Current plan, Admin/User both have previlages to change priority of applications. More API's from Admin and User to be discussed in yarn-3250.

          Show
          rohithsharma Rohith Sharma K S added a comment - Hi Jian He, Do you plan to do client side changes as part of this jira ? YARN-3250 is planning to do changes for admin and user CLI i.e ApplicationClientProtocol. This jira is intended for only scheduler side chagnes support for API's. In YARN-3250 , will be using these exposed API's and implementing it. Current plan, Admin/User both have previlages to change priority of applications. More API's from Admin and User to be discussed in yarn-3250.
          Hide
          sunilg Sunil G added a comment -

          Hi Jian He and Rohith Sharma K S
          We also raised YARN-3970 which is planned use same scheduler apis to get/set priority of an application at runtime from REST. So if we can do a common server side implementation in this ticket, it can be used in others.
          Please share thoughts. Meantime I will update a patch addressing other comments.

          Show
          sunilg Sunil G added a comment - Hi Jian He and Rohith Sharma K S We also raised YARN-3970 which is planned use same scheduler apis to get/set priority of an application at runtime from REST. So if we can do a common server side implementation in this ticket, it can be used in others. Please share thoughts. Meantime I will update a patch addressing other comments.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Hi Sunil G For REST support, proto changes are not done, but for admin/user proto changes to be done. So I mean it can be done separate jira

          Show
          rohithsharma Rohith Sharma K S added a comment - Hi Sunil G For REST support, proto changes are not done, but for admin/user proto changes to be done. So I mean it can be done separate jira
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for the comments. Uploading patch as per the comments.

          Show
          sunilg Sunil G added a comment - Thank you Jian He for the comments. Uploading patch as per the comments.
          Hide
          jianhe Jian He added a comment -

          thanks for updating, looks good overall, few comments:

          • SchedulerApplicationAttempt#appPriority can also be volatile ?
          • testRMRestartWithChangeInPriority- Since the state store update is asynchronous, the test may fail if the state is not updated immediately, we may wait until the state is updated?
          Show
          jianhe Jian He added a comment - thanks for updating, looks good overall, few comments: SchedulerApplicationAttempt#appPriority can also be volatile ? testRMRestartWithChangeInPriority- Since the state store update is asynchronous, the test may fail if the state is not updated immediately, we may wait until the state is updated?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 0s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 44s There were no new javac warning messages.
          +1 javadoc 9m 38s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 49s The applied patch generated 1 new checkstyle issues (total was 181, now 182).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 52m 41s Tests passed in hadoop-yarn-server-resourcemanager.
              92m 43s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747616/0002-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 69b0957
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8698/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8698/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 0s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 49s The applied patch generated 1 new checkstyle issues (total was 181, now 182). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 52m 41s Tests passed in hadoop-yarn-server-resourcemanager.     92m 43s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747616/0002-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 69b0957 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8698/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8698/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8698/console This message was automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          thanks Sunil G for updating patch.
          Some comments

          1. The invocation rmContext.getStateStore().updateApplicationState(appState); is asynchronous. So I feel stil there will be corner case would ocure where priority has set in scheduler but not updated to RMStateSstore. So if any RM switch/Restart would end up in resulting in old priority set. I think this particular invocation should be should be synchronous like any others API's E.g: storeRMDelegationToken, storeRMDTMasterKey
          Show
          rohithsharma Rohith Sharma K S added a comment - thanks Sunil G for updating patch. Some comments The invocation rmContext.getStateStore().updateApplicationState(appState); is asynchronous. So I feel stil there will be corner case would ocure where priority has set in scheduler but not updated to RMStateSstore. So if any RM switch/Restart would end up in resulting in old priority set. I think this particular invocation should be should be synchronous like any others API's E.g: storeRMDelegationToken , storeRMDTMasterKey
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for the comments. I will update a patch addressing the same.
          Rohith Sharma K S If I understood your comment correctly, we may get a scenario when RMStateStore is slow in processing events and RM restart happened. Hence priority will not be updated in state store.

            public synchronized void updateApplicationState(
                ApplicationStateData appState) {
              dispatcher.getEventHandler().handle(new RMStateUpdateAppEvent(appState));
            }
          

          This api is using the event handler to process the new RMStateUpdate. As mentioned by you, If we see storeRMDelegationToken, it calls handleStoreEvent directly.

          So to summarize, could we have a new api in RMStateStore as follows

            public synchronized void updateApplicationStateSynchronizly(
                ApplicationStateData appState) {
              handleStoreEvent(new RMStateUpdateAppEvent(appState));
            }
          

          and use it from scheduler to avoid this case. Thoughts?

          Show
          sunilg Sunil G added a comment - Thank you Jian He for the comments. I will update a patch addressing the same. Rohith Sharma K S If I understood your comment correctly, we may get a scenario when RMStateStore is slow in processing events and RM restart happened. Hence priority will not be updated in state store. public synchronized void updateApplicationState( ApplicationStateData appState) { dispatcher.getEventHandler().handle( new RMStateUpdateAppEvent(appState)); } This api is using the event handler to process the new RMStateUpdate. As mentioned by you, If we see storeRMDelegationToken , it calls handleStoreEvent directly. So to summarize, could we have a new api in RMStateStore as follows public synchronized void updateApplicationStateSynchronizly( ApplicationStateData appState) { handleStoreEvent( new RMStateUpdateAppEvent(appState)); } and use it from scheduler to avoid this case. Thoughts?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Your understanding is correct. I was meant to say to have new synchronous API like updateApplicationStateSynchronizly in RMStateStore.
          Jian He what do you think having new synchronous api in RMstatstore?

          Show
          rohithsharma Rohith Sharma K S added a comment - Your understanding is correct. I was meant to say to have new synchronous API like updateApplicationStateSynchronizly in RMStateStore. Jian He what do you think having new synchronous api in RMstatstore?
          Hide
          jianhe Jian He added a comment -

          sounds good to me

          Show
          jianhe Jian He added a comment - sounds good to me
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He and Rohith Sharma K S for the comments. Updating a latest patch addressing comments.

          Show
          sunilg Sunil G added a comment - Thank you Jian He and Rohith Sharma K S for the comments. Updating a latest patch addressing comments.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Sunil G, I took a quick look at the patch, some comments:

          • I think you should acquire synchronized lock of LeafQueue when reordering schedulableEntity.
          • SchedulerApplication / SchedulerApplicationAttempt.priority should be volatile, please let me know if you think it's not necessary.
          • Have you considered the case when SchedulerApplicationAttempt is not created? If you allow edit SchedulerApplication.priority alone (When SchedulerAttempt is not created), you may need to consider one race condition is: SchedulerApplication changing priority and creating SchedulerAttempt happen together, which may need to acquire CapacityScheduler lock.
          Show
          leftnoteasy Wangda Tan added a comment - Sunil G , I took a quick look at the patch, some comments: I think you should acquire synchronized lock of LeafQueue when reordering schedulableEntity. SchedulerApplication / SchedulerApplicationAttempt.priority should be volatile, please let me know if you think it's not necessary. Have you considered the case when SchedulerApplicationAttempt is not created? If you allow edit SchedulerApplication.priority alone (When SchedulerAttempt is not created), you may need to consider one race condition is: SchedulerApplication changing priority and creating SchedulerAttempt happen together, which may need to acquire CapacityScheduler lock.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 19m 58s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 10m 35s There were no new javac warning messages.
          +1 javadoc 11m 25s There were no new javadoc warning messages.
          +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 36s There were no new checkstyle issues.
          +1 whitespace 0m 7s The patch has no lines that end in whitespace.
          +1 install 1m 43s mvn install still works.
          +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
          +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 53m 4s Tests failed in hadoop-yarn-server-resourcemanager.
              100m 19s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748024/0003-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8acb30b
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8717/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8717/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8717/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 58s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 10m 35s There were no new javac warning messages. +1 javadoc 11m 25s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 36s There were no new checkstyle issues. +1 whitespace 0m 7s The patch has no lines that end in whitespace. +1 install 1m 43s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 53m 4s Tests failed in hadoop-yarn-server-resourcemanager.     100m 19s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748024/0003-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8acb30b hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8717/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8717/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8717/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Thank you Wangda Tan
          Attaching a latest patch addressing the comments.

          Show
          sunilg Sunil G added a comment - Thank you Wangda Tan Attaching a latest patch addressing the comments.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Sunil G thanks for updating patch
          One comment

          1. The below code should not be synchronized. If we have synchronized, then there is very high chance of deadlock. The locking order should be always from stateMachine --> RMStateStore but below code locks in RMStateStore --> stateMachine --> RMStateStore which causes deadlock. For more discussion refer YARN-2946
            +  public synchronized void updateApplicationStateSynchronously(
            +      ApplicationStateData appState) {
            +    handleStoreEvent(new RMStateUpdateAppEvent(appState));
            +  }
            
          Show
          rohithsharma Rohith Sharma K S added a comment - Sunil G thanks for updating patch One comment The below code should not be synchronized. If we have synchronized, then there is very high chance of deadlock. The locking order should be always from stateMachine --> RMStateStore but below code locks in RMStateStore --> stateMachine --> RMStateStore which causes deadlock. For more discussion refer YARN-2946 + public synchronized void updateApplicationStateSynchronously( + ApplicationStateData appState) { + handleStoreEvent( new RMStateUpdateAppEvent(appState)); + }
          Hide
          sunilg Sunil G added a comment -

          Thanks Rohith Sharma K S
          Yes, I agree with you. That extra RMStateStore lock is not needed. I have updated a patch

          Show
          sunilg Sunil G added a comment - Thanks Rohith Sharma K S Yes, I agree with you. That extra RMStateStore lock is not needed. I have updated a patch
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 46s There were no new javac warning messages.
          +1 javadoc 9m 46s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 48s The applied patch generated 2 new checkstyle issues (total was 263, now 265).
          +1 whitespace 0m 5s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 53m 2s Tests passed in hadoop-yarn-server-resourcemanager.
              91m 55s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748137/0004-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 93d50b7
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8734/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8734/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 9m 46s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 48s The applied patch generated 2 new checkstyle issues (total was 263, now 265). +1 whitespace 0m 5s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 53m 2s Tests passed in hadoop-yarn-server-resourcemanager.     91m 55s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748137/0004-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 93d50b7 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8734/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8734/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8734/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 3s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 39s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 46s The applied patch generated 2 new checkstyle issues (total was 263, now 265).
          +1 whitespace 0m 7s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 52m 48s Tests failed in hadoop-yarn-server-resourcemanager.
              90m 45s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748148/0005-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 93d50b7
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8736/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8736/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 3s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 39s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 46s The applied patch generated 2 new checkstyle issues (total was 263, now 265). +1 whitespace 0m 7s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 52m 48s Tests failed in hadoop-yarn-server-resourcemanager.     90m 45s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748148/0005-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 93d50b7 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8736/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8736/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8736/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          This test case is passing locally. Not related to this patch.

          Show
          sunilg Sunil G added a comment - This test case is passing locally. Not related to this patch.
          Hide
          jianhe Jian He added a comment -

          thanks Sunil, for below, I think we can add a appPriorityUpdated method in OrderingPolicy and follow the implementation of demandUpdated method. Doing that, the synchronization of queue may not be needed.

              synchronized (queue) {
                queue.getOrderingPolicy().removeSchedulableEntity(
                    application.getCurrentAppAttempt());
          
                // Update new priority in SchedulerApplication
                application.setPriority(appPriority);
          
                queue.getOrderingPolicy().addSchedulableEntity(
                    application.getCurrentAppAttempt());
              }
          Show
          jianhe Jian He added a comment - thanks Sunil, for below, I think we can add a appPriorityUpdated method in OrderingPolicy and follow the implementation of demandUpdated method. Doing that, the synchronization of queue may not be needed. synchronized (queue) { queue.getOrderingPolicy().removeSchedulableEntity( application.getCurrentAppAttempt()); // Update new priority in SchedulerApplication application.setPriority(appPriority); queue.getOrderingPolicy().addSchedulableEntity( application.getCurrentAppAttempt()); }
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          One comment

          1. TreeSet will throw NullPointerException while adding/removing null object. Suppose, SchedulintApplicationAttempt is not created then application.getCurrentAppAttempt() will be null which would throw NPE. I think this has to be handled in AbstractComparatorOrderingPolicy#removeSchedulableEntity and AbstractComparatorOrderingPolicy#addSchedulableEntity
          Show
          rohithsharma Rohith Sharma K S added a comment - One comment TreeSet will throw NullPointerException while adding/removing null object. Suppose, SchedulintApplicationAttempt is not created then application.getCurrentAppAttempt() will be null which would throw NPE. I think this has to be handled in AbstractComparatorOrderingPolicy#removeSchedulableEntity and AbstractComparatorOrderingPolicy#addSchedulableEntity
          Hide
          sunilg Sunil G added a comment -

          Thank you very much Jian He and Rohith Sharma K S for the comments.
          As discussed with Jian offline, we are keeping current implementation in the patch to avoid more locks within ordering policy. I have addressed the comment from Rohith by providing a null check within the policy.

          Thank you. Uploading a new version of patch. please help to check the same and share the comments.

          Show
          sunilg Sunil G added a comment - Thank you very much Jian He and Rohith Sharma K S for the comments. As discussed with Jian offline, we are keeping current implementation in the patch to avoid more locks within ordering policy. I have addressed the comment from Rohith by providing a null check within the policy. Thank you. Uploading a new version of patch. please help to check the same and share the comments.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 10s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 9m 42s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 49s The applied patch generated 2 new checkstyle issues (total was 266, now 268).
          -1 whitespace 0m 7s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 53m 19s Tests failed in hadoop-yarn-server-resourcemanager.
              91m 38s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.TestRMAdminService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12749454/0006-YARN-3887.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8f73bdd
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/whitespace.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8804/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8804/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 10s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 49s The applied patch generated 2 new checkstyle issues (total was 266, now 268). -1 whitespace 0m 7s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 53m 19s Tests failed in hadoop-yarn-server-resourcemanager.     91m 38s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.TestRMAdminService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749454/0006-YARN-3887.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f73bdd checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/whitespace.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8804/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8804/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8804/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Test failures are unrelated to this patch.

          Show
          sunilg Sunil G added a comment - Test failures are unrelated to this patch.
          Hide
          jianhe Jian He added a comment -

          +1, Rohith Sharma K S, do you have any other comments

          Show
          jianhe Jian He added a comment - +1, Rohith Sharma K S , do you have any other comments
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          +1 lgtm

          Show
          rohithsharma Rohith Sharma K S added a comment - +1 lgtm
          Hide
          jianhe Jian He added a comment -

          Committed to trunk and branch-2, thanks Sunil !
          thanks Rohith, Wangda for reviewing the patch !

          Show
          jianhe Jian He added a comment - Committed to trunk and branch-2, thanks Sunil ! thanks Rohith, Wangda for reviewing the patch !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8282 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8282/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8282 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8282/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #284 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/284/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #284 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/284/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1014 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1014/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1014 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1014/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          Hide
          sunilg Sunil G added a comment -

          Thank you very much Jian He for reviewing and committing the same. Wangda Tan and Rohith Sharma K S, thank you for the reviews.

          Show
          sunilg Sunil G added a comment - Thank you very much Jian He for reviewing and committing the same. Wangda Tan and Rohith Sharma K S , thank you for the reviews.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2230/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2230/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #281 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/281/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #281 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/281/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2211 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2211/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2211 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2211/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/273/)
          YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/273/ ) YARN-3887 . Support changing Application priority during runtime. Contributed by Sunil G (jianhe: rev fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/AbstractComparatorOrderingPolicy.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java

            People

            • Assignee:
              sunilg Sunil G
              Reporter:
              sunilg Sunil G
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development