Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2492 (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests
  3. YARN-2923

Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As part of Distributed Node Labels configuration we need to support Node labels to be configured in Yarn-site.xml. And on modification of Node Labels configuration in yarn-site.xml, NM should be able to get modified Node labels from this NodeLabelsprovider service without NM restart

      1. YARN-2923.20141204-1.patch
        12 kB
        Naganarasimha G R
      2. YARN-2923.20141210-1.patch
        12 kB
        Naganarasimha G R
      3. YARN-2923.20150328-1.patch
        12 kB
        Naganarasimha G R
      4. YARN-2923.20150404-1.patch
        12 kB
        Naganarasimha G R
      5. YARN-2923.20150517-1.patch
        35 kB
        Naganarasimha G R
      6. YARN-2923.20150817-1.patch
        44 kB
        Naganarasimha G R
      7. YARN-2923.20150818-1.patch
        45 kB
        Naganarasimha G R

        Issue Links

          Activity

          Hide
          Naganarasimha Naganarasimha G R added a comment -

          uploading patch after segregating configuration based NodeLabelProvider service from YARN-2495

          Show
          Naganarasimha Naganarasimha G R added a comment - uploading patch after segregating configuration based NodeLabelProvider service from YARN-2495
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Updating the patch after completely removing ConfigurationNodeLabelsProvider from yarn-2495

          Show
          Naganarasimha Naganarasimha G R added a comment - Updating the patch after completely removing ConfigurationNodeLabelsProvider from yarn-2495
          Hide
          john.jian.fang Jian Fang added a comment -

          Hi Naganarasimha, could you please update your patch? I tried to merge in your patch after applied YARN-2495, but could not find where NM_NODE_LABELS_PREFIX and NodeLabelsProviderService are defined. Thanks.

          Show
          john.jian.fang Jian Fang added a comment - Hi Naganarasimha, could you please update your patch? I tried to merge in your patch after applied YARN-2495 , but could not find where NM_NODE_LABELS_PREFIX and NodeLabelsProviderService are defined. Thanks.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Jian Fang.
          Have rebased the patch but this jira and yarn-2729 are both not reviewed, so please be informed that configurations are not finalized and might get change based on review

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Jian Fang . Have rebased the patch but this jira and yarn-2729 are both not reviewed, so please be informed that configurations are not finalized and might get change based on review
          Hide
          john.jian.fang Jian Fang added a comment -

          Thanks Naganarasimha for your quick update.

          I think you should add a flag to turn on/off the periodic check of labels from configuration (should do the same for the script provider). In our use case, the labels are static and we don't like to waste system resources to repeatedly check the configuration file. The node labels are loaded only when the node manager starts up. Does this make sense? Thanks.

          Show
          john.jian.fang Jian Fang added a comment - Thanks Naganarasimha for your quick update. I think you should add a flag to turn on/off the periodic check of labels from configuration (should do the same for the script provider). In our use case, the labels are static and we don't like to waste system resources to repeatedly check the configuration file. The node labels are loaded only when the node manager starts up. Does this make sense? Thanks.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Jian Fang for your feedback, Will check the feasibility during the review of these 2 jiras and try to get your suggestion in, but for now if you are trying to configure only once then you can use RM REST API configuration for Node Labels from your controller when adding additional nodes, Please share your opinion if you see any disadvantages in doing so.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Jian Fang for your feedback, Will check the feasibility during the review of these 2 jiras and try to get your suggestion in, but for now if you are trying to configure only once then you can use RM REST API configuration for Node Labels from your controller when adding additional nodes, Please share your opinion if you see any disadvantages in doing so.
          Hide
          john.jian.fang Jian Fang added a comment -

          That would require that our controller to monitor the lifecycles of the resource manager and the individual nodes, which is what we try to avoid so that our controller will not be tightly coupled with hadoop itself. We still like to load the labels from configuration when node manager starts up. This also avoid information loss when RM is restarted. BTW, would the labels in RM persist some somewhere to handle RM restart and how about the failover in HA case then? Would the labels are reported to both RMs or there is a shared label store (or in the shared RM state store)?

          BTW, seems to me it is not a good idea to periodically check the configuration file. Why not monitor the configuration file by some file watcher instead? Furthermore, HADOOP-7001 defines an API to reload the configuration without restart. Would it possible to integrate HADOOP-7001 with your jira?

          Show
          john.jian.fang Jian Fang added a comment - That would require that our controller to monitor the lifecycles of the resource manager and the individual nodes, which is what we try to avoid so that our controller will not be tightly coupled with hadoop itself. We still like to load the labels from configuration when node manager starts up. This also avoid information loss when RM is restarted. BTW, would the labels in RM persist some somewhere to handle RM restart and how about the failover in HA case then? Would the labels are reported to both RMs or there is a shared label store (or in the shared RM state store)? BTW, seems to me it is not a good idea to periodically check the configuration file. Why not monitor the configuration file by some file watcher instead? Furthermore, HADOOP-7001 defines an API to reload the configuration without restart. Would it possible to integrate HADOOP-7001 with your jira?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12707960/YARN-2923.20150328-1.patch
          against trunk revision 1a495fb.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7159//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707960/YARN-2923.20150328-1.patch against trunk revision 1a495fb. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7159//console This message is automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          cancelling the patch as it required YARN-2729 to go in first

          Show
          Naganarasimha Naganarasimha G R added a comment - cancelling the patch as it required YARN-2729 to go in first
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Rebasing the patch based on changes in yarn-2729
          Jian Fang

          BTW, would the labels in RM persist some somewhere to handle RM restart and how about the failover in HA case then? Would the labels are reported to both RMs or there is a shared label store (or in the shared RM state store)?

          Well in case of distributed configuraton, Every time NM Registers we are sending the Labels, so on restart or HA it gets automatically handled and we are not persisting to a store .

          BTW, seems to me it is not a good idea to periodically check the configuration file. Why not monitor the configuration file by some file watcher instead? Furthermore, HADOOP-7001 defines an API to reload the configuration without restart. Would it possible to integrate HADOOP-7001 with your jira?

          Thanks for the feed back, had a look at the watcher API, generally changes in the file/ directory is identified internally by polling mechanism so i feel its almost as the same approach as we take and it would involve additional effort of finding the path to the yarn-site.xml. and wrt HADOOP-7001, there too, to find whether the configuration is changed or not they do not use watchers and they too do new Configuration, and we do not have multiple configurations and hence felt current interface we have is simpler and lighter but i am still open for modifications if any precise benefit i can see and also would like to get some opinion on this from Tan, Wangda too

          Show
          Naganarasimha Naganarasimha G R added a comment - Rebasing the patch based on changes in yarn-2729 Jian Fang BTW, would the labels in RM persist some somewhere to handle RM restart and how about the failover in HA case then? Would the labels are reported to both RMs or there is a shared label store (or in the shared RM state store)? Well in case of distributed configuraton, Every time NM Registers we are sending the Labels, so on restart or HA it gets automatically handled and we are not persisting to a store . BTW, seems to me it is not a good idea to periodically check the configuration file. Why not monitor the configuration file by some file watcher instead? Furthermore, HADOOP-7001 defines an API to reload the configuration without restart. Would it possible to integrate HADOOP-7001 with your jira? Thanks for the feed back, had a look at the watcher API, generally changes in the file/ directory is identified internally by polling mechanism so i feel its almost as the same approach as we take and it would involve additional effort of finding the path to the yarn-site.xml. and wrt HADOOP-7001 , there too, to find whether the configuration is changed or not they do not use watchers and they too do new Configuration , and we do not have multiple configurations and hence felt current interface we have is simpler and lighter but i am still open for modifications if any precise benefit i can see and also would like to get some opinion on this from Tan, Wangda too
          Hide
          john.jian.fang Jian Fang added a comment -

          Thanks Naganarasimha for your reply.

          I think the watcher API is different from your polling mechanism. I guess it probably uses the Linux inotify mechanism under the hood and the polling only gets back the results from inotify. That could mean the watcher does not actually check the filesystem for each poll. I am not familiar with Java source code and this is only my guess. Here is a discussion on DZone: http://java.dzone.com/news/how-watch-file-system-changes.

          The reason I mentioned HADOOP-7001 is that configuration update is very common in cloud since nodes in a hadoop cluster are ephemeral and could be gone. I know HADOOP-7001 defined some interfaces, but am not aware of its current progress. It would be great if the whole Hadoop project could use the same mechanism to handle configuration updates for different use cases. Does that make sense?

          Show
          john.jian.fang Jian Fang added a comment - Thanks Naganarasimha for your reply. I think the watcher API is different from your polling mechanism. I guess it probably uses the Linux inotify mechanism under the hood and the polling only gets back the results from inotify. That could mean the watcher does not actually check the filesystem for each poll. I am not familiar with Java source code and this is only my guess. Here is a discussion on DZone: http://java.dzone.com/news/how-watch-file-system-changes . The reason I mentioned HADOOP-7001 is that configuration update is very common in cloud since nodes in a hadoop cluster are ephemeral and could be gone. I know HADOOP-7001 defined some interfaces, but am not aware of its current progress. It would be great if the whole Hadoop project could use the same mechanism to handle configuration updates for different use cases. Does that make sense?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Tan, Wangda
          Attaching a WIP patch (may be need some more static checks after jenkins run)

          1. rebased the patch on top of 3565
          2. Moved common code which was earlier in 2729 to here as this jira will be going first
          3. corrected most of Vinod Kumar Vavilapalli's comments in 2729 and pending are :
          • I think the format expected from the command should be more structured. Specifically as we expect more per-label attributes in line with YARN-3565.
          • Not caused by your patch but worth fixing here. NodeStatusUpdaterImpl shouldn't worry about invalid label-set, previous-valid-labels and label validation. You should move all that functionality into NodeLabelsProvider.
          • Can you add the documentation for setting this up too too?

          For these wanted to discuss with you before working on it.

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Tan, Wangda Attaching a WIP patch (may be need some more static checks after jenkins run) rebased the patch on top of 3565 Moved common code which was earlier in 2729 to here as this jira will be going first corrected most of Vinod Kumar Vavilapalli 's comments in 2729 and pending are : I think the format expected from the command should be more structured. Specifically as we expect more per-label attributes in line with YARN-3565 . Not caused by your patch but worth fixing here. NodeStatusUpdaterImpl shouldn't worry about invalid label-set, previous-valid-labels and label validation. You should move all that functionality into NodeLabelsProvider. Can you add the documentation for setting this up too too? For these wanted to discuss with you before working on it.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 38s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 34s There were no new javac warning messages.
          +1 javadoc 9m 35s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 38s The applied patch generated 1 new checkstyle issues (total was 214, now 215).
          +1 whitespace 0m 3s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 47s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          +1 yarn tests 6m 8s Tests passed in hadoop-yarn-server-nodemanager.
              48m 52s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733391/YARN-2923.20150517-1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / ce53c8e
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8016/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8016/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 34s There were no new javac warning messages. +1 javadoc 9m 35s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 38s The applied patch generated 1 new checkstyle issues (total was 214, now 215). +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 47s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 21s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. +1 yarn tests 6m 8s Tests passed in hadoop-yarn-server-nodemanager.     48m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733391/YARN-2923.20150517-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ce53c8e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8016/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8016/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8016/console This message was automatically generated.
          Hide
          dian.fu Dian Fu added a comment -

          Hi Naganarasimha G R,
          Sorry to come to this JIRA late. Thanks a lot for working on this. Most codes LGTM. Just have the following rough thoughts:
          1. Classes such as AbstractNodeLabelsProvider are located in directory hadoop-yarn-server-nodemanager, I'm think if we can move it to hadoop-yarn-server-common. This would ease the work of adding similar node label provider mechanism at RM side in the future if possible. There may need some minor code reactor work to do, such as it accesses NM side configuration NM_NODE_LABELS_PROVIDER_FETCH_INTERVAL_MS.
          2. For YARN-2923 and YARN-2729, both ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider have a mechanism to periodically execute a script or read configuration to fetch labels. Is it possible to merge ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider into one class? This class can provide the function to periodically fetch labels and the actual work of how to fetch labels can be configured in a plugable way. The benefits of this approach are: a) remove duplicate code in ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider; b) ease third parts to add their own node label provider.

          Not sure if the above thoughts are clear and make sense to you? I can help to do some POC based on YARN-2923 and YARN-2729 if needed.

          Show
          dian.fu Dian Fu added a comment - Hi Naganarasimha G R , Sorry to come to this JIRA late. Thanks a lot for working on this. Most codes LGTM. Just have the following rough thoughts: 1. Classes such as AbstractNodeLabelsProvider are located in directory hadoop-yarn-server-nodemanager , I'm think if we can move it to hadoop-yarn-server-common . This would ease the work of adding similar node label provider mechanism at RM side in the future if possible. There may need some minor code reactor work to do, such as it accesses NM side configuration NM_NODE_LABELS_PROVIDER_FETCH_INTERVAL_MS . 2. For YARN-2923 and YARN-2729 , both ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider have a mechanism to periodically execute a script or read configuration to fetch labels. Is it possible to merge ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider into one class? This class can provide the function to periodically fetch labels and the actual work of how to fetch labels can be configured in a plugable way. The benefits of this approach are: a) remove duplicate code in ScriptBasedNodeLabelsProvider and ConfigurationNodeLabelsProvider ; b) ease third parts to add their own node label provider. Not sure if the above thoughts are clear and make sense to you? I can help to do some POC based on YARN-2923 and YARN-2729 if needed.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Dian Fu, for the review. But would also would like to get inputs from other folks in the Opensource for exposing this interface in RM side... may be based on this i would like to move into hadoop-yarn-server-common.
          Wangda Tan, Its been a long time we revisited the distributed node labeling jira's can you please check and review once ...

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Dian Fu , for the review. But would also would like to get inputs from other folks in the Opensource for exposing this interface in RM side... may be based on this i would like to move into hadoop-yarn-server-common . Wangda Tan , Its been a long time we revisited the distributed node labeling jira's can you please check and review once ...
          Hide
          dian.fu Dian Fu added a comment -

          But would also would like to get inputs from other folks in the Opensource for exposing this interface in RM side... may be based on this i would like to move into hadoop-yarn-server-common.

          Yes, of course.

          Show
          dian.fu Dian Fu added a comment - But would also would like to get inputs from other folks in the Opensource for exposing this interface in RM side... may be based on this i would like to move into hadoop-yarn-server-common. Yes, of course.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for updating, Naganarasimha G R! Some comments:

          1) All script provider related configurations/logics should be removed. They should come with different patch.
          Such as: case YarnConfiguration.SCRIPT_NODE_LABELS_PROVIDER:
          And public static final String SCRIPT_NODE_LABELS_PROVIDER = "script";
          Should be removed

          2) For logics in NodeStatusUpdater. Things in my mind are:

          • PreviousNodeLabels will be reset everytime if we do a fetch. (To avoid handle same node labels as much as possible)
          • Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it)
          • Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label)

          A little cosmetic suggestion. I found startStatusUpdater is too complex, full of try catch, etc. I suggest to make label related logic to be: a. Fetch label, check. b. handle response from RM and post process. Each of them should a separated method to improve readability.

          I suggest to keep provider within nodemanager (instead of yarn-server-common) for this patch, we can move it if we decide to do that in the future.

          Please let me know about your thoughts.

          Wangda

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for updating, Naganarasimha G R ! Some comments: 1) All script provider related configurations/logics should be removed. They should come with different patch. Such as: case YarnConfiguration.SCRIPT_NODE_LABELS_PROVIDER: And public static final String SCRIPT_NODE_LABELS_PROVIDER = "script"; Should be removed 2) For logics in NodeStatusUpdater. Things in my mind are: PreviousNodeLabels will be reset everytime if we do a fetch. (To avoid handle same node labels as much as possible) Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it) Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label) A little cosmetic suggestion. I found startStatusUpdater is too complex, full of try catch, etc. I suggest to make label related logic to be: a. Fetch label, check. b. handle response from RM and post process. Each of them should a separated method to improve readability. I suggest to keep provider within nodemanager (instead of yarn-server-common) for this patch, we can move it if we decide to do that in the future. Please let me know about your thoughts. Wangda
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Wangda Tan, for the feedback, but had few things to discuss.

          1) All script provider related configurations/logic should be removed.

          Will take care of it, earlier thought at-least white-list entities let it be there but not a problem will correct it.

          PreviousNodeLabels will be reset every time if we do a fetch. (To avoid handle same node labels as much as possible)
          Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label)

          May be I dint get your approach completely here ? but based on earlier discussions in 2495 we had finalized that we will send the node labels only when there is a change because

          1. unnecessary Network traffic which is amplified in a large cluster which has lot of NM's
          2. Either RM needs to do the validation of whether the labels are modified or blindly update the NodeLabelsStore. In former case is not better because RM will be overloaded to do this operation and in later case unnecessary Store file updates, i feel the magnitude of updates which will be happening in the large cluster with heartbeat coming in every second is too high and all are unwanted updates.
            Based on the above points i feel its better to check and update RM only when there is a change in the NM's node labels

          Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it)

          IIUC this was the conclusion we had in YARN-2495 in the following comment. which i feel is correct because assume a case where in label is based on java version(/lib version) and java version is upgraded in a given NM but the centralized labels new java version is not yet added then NM will fail to update the node labels but if we do not reset it, RM will maintain the NM's NodeLabel as of Prev java version and the client app if run on this node it might fail due to this.

          A little cosmetic suggestion.

          Hmm agree on this but approach i felt we can have a inner class like NMDistributedNodeLabelsHandler and that can maintain the state of previous labels and takes care of initializing and taking care of all methods and logging related to NodeLabels . I have already added some methods but we require to maintain some state of labels in the heartbeat flow so introducing a class like this and pushing all methods related to labels there would be much better and more readable, Thoughts ?

          I suggest to keep provider within nodemanager

          ok

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Wangda Tan , for the feedback, but had few things to discuss. 1) All script provider related configurations/logic should be removed. Will take care of it, earlier thought at-least white-list entities let it be there but not a problem will correct it. PreviousNodeLabels will be reset every time if we do a fetch. (To avoid handle same node labels as much as possible) Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label) May be I dint get your approach completely here ? but based on earlier discussions in 2495 we had finalized that we will send the node labels only when there is a change because unnecessary Network traffic which is amplified in a large cluster which has lot of NM's Either RM needs to do the validation of whether the labels are modified or blindly update the NodeLabelsStore. In former case is not better because RM will be overloaded to do this operation and in later case unnecessary Store file updates, i feel the magnitude of updates which will be happening in the large cluster with heartbeat coming in every second is too high and all are unwanted updates. Based on the above points i feel its better to check and update RM only when there is a change in the NM's node labels Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it) IIUC this was the conclusion we had in YARN-2495 in the following comment . which i feel is correct because assume a case where in label is based on java version(/lib version) and java version is upgraded in a given NM but the centralized labels new java version is not yet added then NM will fail to update the node labels but if we do not reset it, RM will maintain the NM's NodeLabel as of Prev java version and the client app if run on this node it might fail due to this. A little cosmetic suggestion. Hmm agree on this but approach i felt we can have a inner class like NMDistributedNodeLabelsHandler and that can maintain the state of previous labels and takes care of initializing and taking care of all methods and logging related to NodeLabels . I have already added some methods but we require to maintain some state of labels in the heartbeat flow so introducing a class like this and pushing all methods related to labels there would be much better and more readable, Thoughts ? I suggest to keep provider within nodemanager ok
          Hide
          leftnoteasy Wangda Tan added a comment -

          Hi Naga,

          PreviousNodeLabels will be reset every time if we do a fetch. (To avoid handle same node labels as much as possible)

          I cannot remember what I meant previously, so please ignore it.

          Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label)

          Re-read your patch, it should work properly, ignore this comment also.

          Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it)

          What I meant for this one is, in your patch, it reset node label for invalid labels to empty-string-set. I think this is discussible. When a node generated a wrong set of labels (and previous labels are valid), I prefer to keep it unchanged instead of reset labels to empty. IAW, reset invalid labels to empty and send to RM seems a little over-kill to me.

              previousNodeLabels = nodeLabelsForHeartbeat;
              nodeLabelsForHeartbeat =
                  CommonNodeLabelsManager.EMPTY_NODELABEL_SET;
              invalidLabels = true;
          

          This is not conflict with https://issues.apache.org/jira/browse/YARN-2495?focusedCommentId=14358109&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14358109. The comment in YARN-2495 is majorly for registeration, set to empty is correct when we don't have a previously correct labels.

          Hmm agree on this but approach i felt we can have a inner class like NMDistributedNodeLabelsHandler...

          It should be good.

          Thanks,
          Wangda

          Show
          leftnoteasy Wangda Tan added a comment - Hi Naga, PreviousNodeLabels will be reset every time if we do a fetch. (To avoid handle same node labels as much as possible) I cannot remember what I meant previously, so please ignore it. Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid handle same node label) Re-read your patch, it should work properly, ignore this comment also. Don't reset node label if fetched node label is incorrect. (This should be a part of error handling, we should treat it's a error to be avoided instead of force reset it) What I meant for this one is, in your patch, it reset node label for invalid labels to empty-string-set. I think this is discussible. When a node generated a wrong set of labels (and previous labels are valid), I prefer to keep it unchanged instead of reset labels to empty. IAW, reset invalid labels to empty and send to RM seems a little over-kill to me. previousNodeLabels = nodeLabelsForHeartbeat; nodeLabelsForHeartbeat = CommonNodeLabelsManager.EMPTY_NODELABEL_SET; invalidLabels = true ; This is not conflict with https://issues.apache.org/jira/browse/YARN-2495?focusedCommentId=14358109&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14358109 . The comment in YARN-2495 is majorly for registeration, set to empty is correct when we don't have a previously correct labels. Hmm agree on this but approach i felt we can have a inner class like NMDistributedNodeLabelsHandler... It should be good. Thanks, Wangda
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 43s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 43s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 3s The patch has no lines that end in whitespace.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 0m 21s Tests failed in hadoop-yarn-api.
          -1 yarn tests 1m 54s Tests failed in hadoop-yarn-common.
          +1 yarn tests 6m 12s Tests passed in hadoop-yarn-server-nodemanager.
              53m 54s  



          Reason Tests
          Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.util.TestRackResolver



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733391/YARN-2923.20150517-1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8f73bdd
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8794/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8794/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 43s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 43s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 50s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 0m 21s Tests failed in hadoop-yarn-api. -1 yarn tests 1m 54s Tests failed in hadoop-yarn-common. +1 yarn tests 6m 12s Tests passed in hadoop-yarn-server-nodemanager.     53m 54s   Reason Tests Failed unit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.util.TestRackResolver Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733391/YARN-2923.20150517-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f73bdd checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8794/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8794/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8794/console This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Wangda Tan,
          Thanks for the comments,

          I prefer to keep it unchanged instead of reset labels to empty. IAW, reset invalid labels to empty and send to RM seems a little over-kill to me.

          Yes this is debatable topic when we consider the partition labels then seems like not required but when it comes to constraints i think it would be required as per the earlier examples given. One thought i had was, may be for constraints we can group them and one of the value in the group can be given and on invalid labels, labels related to that group can be removed. As there is one more jira (YARN-3506) for error handling thought of further discussing this topic there and as far as this jira i will send nulls on NM Labels validation failure.
          Also have handled as discussed for NMDistributedNodeLabelsHandler.
          TestYarnConfigurationFields test case failure has been corrected and TestRackResolver seems to be not related to this jira and locally its passing.
          Checkstyle issue is related to the number of lines in YarnConfiguration,

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Wangda Tan , Thanks for the comments, I prefer to keep it unchanged instead of reset labels to empty. IAW, reset invalid labels to empty and send to RM seems a little over-kill to me. Yes this is debatable topic when we consider the partition labels then seems like not required but when it comes to constraints i think it would be required as per the earlier examples given. One thought i had was, may be for constraints we can group them and one of the value in the group can be given and on invalid labels, labels related to that group can be removed. As there is one more jira ( YARN-3506 ) for error handling thought of further discussing this topic there and as far as this jira i will send nulls on NM Labels validation failure. Also have handled as discussed for NMDistributedNodeLabelsHandler . TestYarnConfigurationFields test case failure has been corrected and TestRackResolver seems to be not related to this jira and locally its passing. Checkstyle issue is related to the number of lines in YarnConfiguration,
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 57s There were no new javac warning messages.
          +1 javadoc 10m 2s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 44s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 7s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse.
          +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          -1 yarn tests 1m 56s Tests failed in hadoop-yarn-common.
          -1 yarn tests 6m 16s Tests failed in hadoop-yarn-server-nodemanager.
              55m 9s  



          Reason Tests
          Failed unit tests hadoop.yarn.util.TestRackResolver
            hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750760/YARN-2923.20150817-1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 13604bd
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8859/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8859/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 2s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 44s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 7s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse. +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. -1 yarn tests 1m 56s Tests failed in hadoop-yarn-common. -1 yarn tests 6m 16s Tests failed in hadoop-yarn-server-nodemanager.     55m 9s   Reason Tests Failed unit tests hadoop.yarn.util.TestRackResolver   hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750760/YARN-2923.20150817-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 13604bd checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8859/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8859/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8859/console This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Test cases are passing locally multiple times. Adding a patch to get more logs for TestNodeStatusUpdaterForLabels

          Show
          Naganarasimha Naganarasimha G R added a comment - Test cases are passing locally multiple times. Adding a patch to get more logs for TestNodeStatusUpdaterForLabels
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 18m 8s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 39s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 9s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          -1 yarn tests 1m 56s Tests failed in hadoop-yarn-common.
          +1 yarn tests 6m 24s Tests passed in hadoop-yarn-server-nodemanager.
              53m 23s  



          Reason Tests
          Failed unit tests hadoop.yarn.util.TestRackResolver



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750943/YARN-2923.20150818-1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 71566e2
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8871/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8871/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 8s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 39s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 9s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. -1 yarn tests 1m 56s Tests failed in hadoop-yarn-common. +1 yarn tests 6m 24s Tests passed in hadoop-yarn-server-nodemanager.     53m 23s   Reason Tests Failed unit tests hadoop.yarn.util.TestRackResolver Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750943/YARN-2923.20150818-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 71566e2 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8871/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8871/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8871/console This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Wangda Tan,
          Can you take look after the latest patch seems like jenkins result is fine (TestRackResolver is not related to this jira and Checkstyle issue is related to the number of lines in YarnConfiguration)

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Wangda Tan , Can you take look after the latest patch seems like jenkins result is fine (TestRackResolver is not related to this jira and Checkstyle issue is related to the number of lines in YarnConfiguration)
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for update, Naganarasimha G R, latest patch LGTM, yarn-default.xml has wrong indent, I will fix them while commit.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for update, Naganarasimha G R , latest patch LGTM, yarn-default.xml has wrong indent, I will fix them while commit.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Naganarasimha G R, committed to trunk/branch-2.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Naganarasimha G R , committed to trunk/branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8329 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8329/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8329 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8329/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/CHANGES.txt
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Wangda Tan & Vinod Kumar Vavilapalli, for review and committing this jira !

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Wangda Tan & Vinod Kumar Vavilapalli , for review and committing this jira !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/291/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/291/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1024 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1024/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1024 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1024/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #294 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/294/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #294 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/294/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2240 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2240/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2240 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2240/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #283 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/283/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #283 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/283/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2221/)
          YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2221/ ) YARN-2923 . Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) (wangda: rev fc07464d1a48b0413da5e921614430e41263fdb7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForLabels.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java

            People

            • Assignee:
              Naganarasimha Naganarasimha G R
              Reporter:
              Naganarasimha Naganarasimha G R
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development