Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3464

Race condition in LocalizerRunner kills localizer before localizing all resources

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Race condition in LocalizerRunner causes container localization timeout.
      Currently LocalizerRunner will kill the ContainerLocalizer when pending list for LocalizerResourceRequestEvent is empty.

            } else if (pending.isEmpty()) {
              action = LocalizerAction.DIE;
            }
      

      If a LocalizerResourceRequestEvent is added after LocalizerRunner kill the ContainerLocalizer due to empty pending list, this LocalizerResourceRequestEvent will never be handled.
      Without ContainerLocalizer, LocalizerRunner#update will never be called.
      The container will stay at LOCALIZING state, until the container is killed by AM due to TASK_TIMEOUT.

      1. YARN-3464.000.patch
        11 kB
        zhihai xu
      2. YARN-3464.001.patch
        13 kB
        zhihai xu
      3. YARN-3464-branch-2.6.1.txt
        15 kB
        Vinod Kumar Vavilapalli

        Activity

        Hide
        zxu zhihai xu added a comment -

        This issue only happened for PRIVATE/APPLICATION resource Localization
        We saw this issue happened when the PRIVATE LocalizerResourceRequestEvent interleaved with PUBLIC LocalizerResourceRequestEvent in the following order:
        PRIVATE1
        PRIVATE2
        ..........
        PRIVATEm
        PUBLIC1
        PUBLIC2
        .....
        PUBLICn
        PRIVATEm+1
        PRIVATEm+2
        The last two PRIVATE LocalizerResourceRequestEvent is added after all previous m PRIVATE LocalizerResourceRequestEvent are LOCALIZED due to the delay to process n PUBLIC LocalizerResourceRequestEvent.
        Then the container will stay at LOCALIZING state until it is killed by AM.

        Show
        zxu zhihai xu added a comment - This issue only happened for PRIVATE/APPLICATION resource Localization We saw this issue happened when the PRIVATE LocalizerResourceRequestEvent interleaved with PUBLIC LocalizerResourceRequestEvent in the following order: PRIVATE1 PRIVATE2 .......... PRIVATEm PUBLIC1 PUBLIC2 ..... PUBLICn PRIVATEm+1 PRIVATEm+2 The last two PRIVATE LocalizerResourceRequestEvent is added after all previous m PRIVATE LocalizerResourceRequestEvent are LOCALIZED due to the delay to process n PUBLIC LocalizerResourceRequestEvent. Then the container will stay at LOCALIZING state until it is killed by AM.
        Hide
        kasha Karthik Kambatla added a comment -

        I have been investigating a similar issue. Initially I thought of the same race, but not sure if that alone solves the issue.

        Looking at the code closely, I don't see any resources being removed from pending. So, pending shouldn't be empty after some of the resources have been downloaded.

        Related: YARN-3024 increases the frequency of this issue.

        Show
        kasha Karthik Kambatla added a comment - I have been investigating a similar issue. Initially I thought of the same race, but not sure if that alone solves the issue. Looking at the code closely, I don't see any resources being removed from pending. So, pending shouldn't be empty after some of the resources have been downloaded. Related: YARN-3024 increases the frequency of this issue.
        Hide
        kasha Karthik Kambatla added a comment -

        Looking at the code closely, I don't see any resources being removed from pending. So, pending shouldn't be empty after some of the resources have been downloaded.

        Never mind. findNextResource has a call to iterator.remove().

        In any case, I think the right approach is to send an explicit event to the localizer to indicate we are done with localizing all the resources. On receiving this, the localizer tracker sends the DIE action.

        Show
        kasha Karthik Kambatla added a comment - Looking at the code closely, I don't see any resources being removed from pending. So, pending shouldn't be empty after some of the resources have been downloaded. Never mind. findNextResource has a call to iterator.remove(). In any case, I think the right approach is to send an explicit event to the localizer to indicate we are done with localizing all the resources. On receiving this, the localizer tracker sends the DIE action.
        Hide
        zxu zhihai xu added a comment -

        Karthik Kambatla, thanks for the information. I just looked at YARN-3024, Yes, it will make this issue happen more frequently.
        Before YARN-3024, The localization for private resource is one by one. The next one won't start until the current one finish localization.
        It will take longer time for private resource localization.
        With YARN-3024, The localization will be done in parallel, multiple files can be localized at the same time.
        The chance for ContainerLocalizer being killed when the last two PRIVATE LocalizerResourceRequestEvent are added is bigger.
        Yes, your suggestion is also what I thought.

        Show
        zxu zhihai xu added a comment - Karthik Kambatla , thanks for the information. I just looked at YARN-3024 , Yes, it will make this issue happen more frequently. Before YARN-3024 , The localization for private resource is one by one. The next one won't start until the current one finish localization. It will take longer time for private resource localization. With YARN-3024 , The localization will be done in parallel, multiple files can be localized at the same time. The chance for ContainerLocalizer being killed when the last two PRIVATE LocalizerResourceRequestEvent are added is bigger. Yes, your suggestion is also what I thought.
        Hide
        zxu zhihai xu added a comment -

        I also created another JIRA YARN-3465, which can help this issue and make sure localization is based on the correct order:
        PUBLIC, PRIVATE and APPLICATION.
        The issue in my case is also because PRIVATE LocalResourceRequest is reordered to first and APPLICATION LocalResourceRequest is reordered to last. The PUBLIC LocalResourceRequest is in the middle which add delay for APPLICATION LocalResourceRequest.
        Because the entrySet order based on HashMap will not be fixed. use LinkedHashMap should be used.

        Show
        zxu zhihai xu added a comment - I also created another JIRA YARN-3465 , which can help this issue and make sure localization is based on the correct order: PUBLIC, PRIVATE and APPLICATION. The issue in my case is also because PRIVATE LocalResourceRequest is reordered to first and APPLICATION LocalResourceRequest is reordered to last. The PUBLIC LocalResourceRequest is in the middle which add delay for APPLICATION LocalResourceRequest. Because the entrySet order based on HashMap will not be fixed. use LinkedHashMap should be used.
        Hide
        kasha Karthik Kambatla added a comment -

        We can may be discuss this more on YARN-3465, but I don't think having it sorted is necessary. The container can not be started until all the resources are localized; so, the order of their downloads shouldn't matter as long as they all get localized.

        Show
        kasha Karthik Kambatla added a comment - We can may be discuss this more on YARN-3465 , but I don't think having it sorted is necessary. The container can not be started until all the resources are localized; so, the order of their downloads shouldn't matter as long as they all get localized.
        Hide
        kasha Karthik Kambatla added a comment -

        Discussed this with Zhihai offline.

        Firstly, responding with a DIE when LocalizerRunner#pending is empty seems wrong. I don't think synchronizing that check is really going to help if someone can add a request to pending after that. We see two alternatives:

        1. If we are going to leave the ContainerLocalizer around, why do we even bother issuing a DIE action? Can we just let it LIVE forever.
        2. If there is merit to killing the ContainerLocalizer, we could have a sentinel sent for each localizer. DIE action is to be sent only when either of the two conditions are met:
          1. One of the localizations failed. Since the container can't be launched, kill the localizer and fail the container.
          2. The sentinel has been received and all previously scheduled localizations have been fetched successfully (LocalizerRunner#scheduled.isEmpty())

        While we make these changes, I see how YARN-3465 could be useful in reducing the likelihood of this issue in the interim.

        Show
        kasha Karthik Kambatla added a comment - Discussed this with Zhihai offline. Firstly, responding with a DIE when LocalizerRunner#pending is empty seems wrong. I don't think synchronizing that check is really going to help if someone can add a request to pending after that. We see two alternatives: If we are going to leave the ContainerLocalizer around, why do we even bother issuing a DIE action? Can we just let it LIVE forever. If there is merit to killing the ContainerLocalizer, we could have a sentinel sent for each localizer. DIE action is to be sent only when either of the two conditions are met: One of the localizations failed. Since the container can't be launched, kill the localizer and fail the container. The sentinel has been received and all previously scheduled localizations have been fetched successfully ( LocalizerRunner#scheduled.isEmpty() ) While we make these changes, I see how YARN-3465 could be useful in reducing the likelihood of this issue in the interim.
        Hide
        zxu zhihai xu added a comment -

        Thanks Karthik Kambatla for the detail explanation.
        For the suggested first solution,
        ContainerLocalizer will be killed by ContainerImpl#cleanup when the container is finished.
        For the suggested second solution,
        ContainerLocalizer can be killed earlier by a sentinel/event sent by ContainerImp after localization before launch container.
        I want to check whether other people has better idea for this issue before I implement a patch for this issue.

        Show
        zxu zhihai xu added a comment - Thanks Karthik Kambatla for the detail explanation. For the suggested first solution, ContainerLocalizer will be killed by ContainerImpl#cleanup when the container is finished. For the suggested second solution, ContainerLocalizer can be killed earlier by a sentinel/event sent by ContainerImp after localization before launch container. I want to check whether other people has better idea for this issue before I implement a patch for this issue.
        Hide
        kasha Karthik Kambatla added a comment -

        Jason Lowe - do you think any of my earlier suggestions are reasonable?

        Show
        kasha Karthik Kambatla added a comment - Jason Lowe - do you think any of my earlier suggestions are reasonable?
        Hide
        jlowe Jason Lowe added a comment -

        We shouldn't leave a ContainerLocalizer lingering around if the container is ready to be launched, as that's wasting node resources and adding extra localizer heartbeat processing on the NM we don't need to do. One exception to that would be if we want to support localizing new resources while a container is already running, but last I checked we don't support that.

        IMHO it makes sense to kill the localizer when the container is ready to be launched. If it's not ready to be launched then we may need to (re)localize some resource and the localizer would have some utility to keep running. So I'd look into changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched."

        Show
        jlowe Jason Lowe added a comment - We shouldn't leave a ContainerLocalizer lingering around if the container is ready to be launched, as that's wasting node resources and adding extra localizer heartbeat processing on the NM we don't need to do. One exception to that would be if we want to support localizing new resources while a container is already running, but last I checked we don't support that. IMHO it makes sense to kill the localizer when the container is ready to be launched. If it's not ready to be launched then we may need to (re)localize some resource and the localizer would have some utility to keep running. So I'd look into changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched."
        Hide
        kasha Karthik Kambatla added a comment -

        So I'd look into changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched."

        Thanks Jason Lowe. Yes, that would simplify the localizer-NM communication significantly.

        We could add the kill-container-localizer logic to either LocalizedTransition or LaunchTransition in ContainerImpl. I prefer we do it in the LocalizedTransition.

        Show
        kasha Karthik Kambatla added a comment - So I'd look into changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched." Thanks Jason Lowe . Yes, that would simplify the localizer-NM communication significantly. We could add the kill-container-localizer logic to either LocalizedTransition or LaunchTransition in ContainerImpl. I prefer we do it in the LocalizedTransition .
        Hide
        zxu zhihai xu added a comment -

        thanks Jason Lowe and Karthik Kambatla,

        changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched."

        That is a fantastic summary. I will implement the patch based on this.

        Show
        zxu zhihai xu added a comment - thanks Jason Lowe and Karthik Kambatla , changing the logic from "kill me when there's no more work in my queue" to "kill me when my container is ready to be launched." That is a fantastic summary. I will implement the patch based on this.
        Hide
        zxu zhihai xu added a comment -

        I uploaded a patch YARN-3464.000.patch for review.

        Show
        zxu zhihai xu added a comment - I uploaded a patch YARN-3464 .000.patch for review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 49s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 38s There were no new javac warning messages.
        -1 javadoc 9m 52s The applied patch generated 50 additional warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 4m 4s The applied patch generated 3 additional checkstyle issues.
        +1 install 1m 38s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        -1 yarn tests 5m 44s Tests failed in hadoop-yarn-server-nodemanager.
            45m 46s  



        Reason Tests
        Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12727047/YARN-3464.000.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / cfba355
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/diffJavadocWarnings.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7439/testReport/
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7439//console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 49s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 38s There were no new javac warning messages. -1 javadoc 9m 52s The applied patch generated 50 additional warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 4m 4s The applied patch generated 3 additional checkstyle issues. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 yarn tests 5m 44s Tests failed in hadoop-yarn-server-nodemanager.     45m 46s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727047/YARN-3464.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / cfba355 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7439/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7439/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7439//console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 48s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 1s The patch appears to include 1 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 45s There were no new javac warning messages.
        +1 javadoc 9m 31s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 5m 29s The applied patch generated 3 additional checkstyle issues.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 1s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        -1 yarn tests 5m 44s Tests failed in hadoop-yarn-server-nodemanager.
            46m 51s  



        Reason Tests
        Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12727091/YARN-3464.000.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e71d0d8
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7442/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7442/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7442/testReport/
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7442//console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 48s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 1s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 9m 31s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 29s The applied patch generated 3 additional checkstyle issues. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 1s The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 yarn tests 5m 44s Tests failed in hadoop-yarn-server-nodemanager.     46m 51s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727091/YARN-3464.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e71d0d8 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7442/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7442/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7442/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7442//console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 30s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 33s There were no new javac warning messages.
        +1 javadoc 9m 31s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 5m 27s The applied patch generated 3 additional checkstyle issues.
        +1 install 1m 40s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 yarn tests 6m 0s Tests passed in hadoop-yarn-server-nodemanager.
            46m 46s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12727117/YARN-3464.000.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 674c7ef
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7443/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7443/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7443/testReport/
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7443//console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 30s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 31s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 27s The applied patch generated 3 additional checkstyle issues. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 6m 0s Tests passed in hadoop-yarn-server-nodemanager.     46m 46s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727117/YARN-3464.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 674c7ef checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7443/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7443/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7443/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7443//console This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks for uploading the patch, zhihai xu.

        Comments:

        1. ContainerImpl: unrelated to the patch, container.metrics.endInitingContainer() should likely move up to right after container.sendLaunchEvent()
        2. Nit: Simplify javadoc for ResourceLocalizationService#handleContainerResourcesLocalized. How about "Once a container's resources are localized, kill the corresponding {@link ContainerLocalizer}

          "

        3. ResourceLocalizationService.LocalizerRunner#update - mostly unrelated to this patch
          1. Rename to processHeartbeat?
          2. I don't see the point of a LocalizerAction variable initialized to DIE. How about changing that to a boolean fetchFailed?
        Show
        kasha Karthik Kambatla added a comment - Thanks for uploading the patch, zhihai xu . Comments: ContainerImpl: unrelated to the patch, container.metrics.endInitingContainer() should likely move up to right after container.sendLaunchEvent() Nit: Simplify javadoc for ResourceLocalizationService#handleContainerResourcesLocalized . How about "Once a container's resources are localized, kill the corresponding {@link ContainerLocalizer} " ResourceLocalizationService.LocalizerRunner#update - mostly unrelated to this patch Rename to processHeartbeat ? I don't see the point of a LocalizerAction variable initialized to DIE. How about changing that to a boolean fetchFailed ?
        Hide
        zxu zhihai xu added a comment -

        Karthik Kambatla, thanks for the review. Your suggestions are very reasonable, which make the code much easier to read.
        I uploaded a new patch YARN-3464.001.patch, which addressed all your comments. Please review it.

        Show
        zxu zhihai xu added a comment - Karthik Kambatla , thanks for the review. Your suggestions are very reasonable, which make the code much easier to read. I uploaded a new patch YARN-3464 .001.patch, which addressed all your comments. Please review it.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 55s There were no new javac warning messages.
        +1 javadoc 9m 49s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 5m 30s The applied patch generated 3 additional checkstyle issues.
        +1 install 1m 37s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 yarn tests 5m 51s Tests passed in hadoop-yarn-server-nodemanager.
            47m 46s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12728198/YARN-3464.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a00e001
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7503/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7503/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7503/testReport/
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7503/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 30s The applied patch generated 3 additional checkstyle issues. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 5m 51s Tests passed in hadoop-yarn-server-nodemanager.     47m 46s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728198/YARN-3464.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a00e001 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7503/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7503/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7503/testReport/ Console output https://builds.apache.org/job/PreCommit-YARN-Build/7503/console This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        +1

        Show
        kasha Karthik Kambatla added a comment - +1
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7679 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7679/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7679 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7679/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/CHANGES.txt
        Hide
        kasha Karthik Kambatla added a comment -

        Just committed to trunk and branch-2.

        Thanks zhihai xu for the patch, and Jason Lowe for your inputs.

        Show
        kasha Karthik Kambatla added a comment - Just committed to trunk and branch-2. Thanks zhihai xu for the patch, and Jason Lowe for your inputs.
        Hide
        zxu zhihai xu added a comment -

        thanks Karthik Kambatla for the review and committing the patch, thanks Jason Lowe for the valuable feedback.

        Show
        zxu zhihai xu added a comment - thanks Karthik Kambatla for the review and committing the patch, thanks Jason Lowe for the valuable feedback.
        Hide
        jira.shegalov Gera Shegalov added a comment -

        We might need to tweak checkstyle rules. There are a bunch of 80-column-limit violations that seem come from the import statements.

        Show
        jira.shegalov Gera Shegalov added a comment - We might need to tweak checkstyle rules. There are a bunch of 80-column-limit violations that seem come from the import statements.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2108 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2108/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2108 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2108/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #167 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/167/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #167 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/167/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #910 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/910/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #910 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/910/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #176 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/176/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #176 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/176/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        Hide
        jlowe Jason Lowe added a comment -

        Given this is reportedly much more frequent with YARN-3024, is there a reason we don't want this fixed in 2.7.1?

        Show
        jlowe Jason Lowe added a comment - Given this is reportedly much more frequent with YARN-3024 , is there a reason we don't want this fixed in 2.7.1?
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/177/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/177/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2126 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2126/)
        YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2126 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2126/ ) YARN-3464 . Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) (kasha: rev 47279c3228185548ed09c36579b420225e4894f5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizationEventType.java
        Hide
        kasha Karthik Kambatla added a comment -

        Committed to branch-2.7 as well. Jason Lowe - thanks for the ping.

        Show
        kasha Karthik Kambatla added a comment - Committed to branch-2.7 as well. Jason Lowe - thanks for the ping.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7688 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7688/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7688 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7688/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2109 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2109 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/177/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #177 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/177/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #911 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/911/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #911 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/911/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/178/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/178/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2127 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2127/)
        Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e)

        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2127 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2127/ ) Update CHANGES.txt - Pulled in YARN-3465 , YARN-3516 , and YARN-3464 to branch-2.7 (for 2.7.1) (kasha: rev 32cd2c8d429ddb87348299c00b7d851246a25b4e) hadoop-yarn-project/CHANGES.txt
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Pulled this into 2.6.1. Patch had 3 merge conflicts, fixed them.

        Ran compilation and TestResourceLocalizationService before the push.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1. Patch had 3 merge conflicts, fixed them. Ran compilation and TestResourceLocalizationService before the push.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Attaching the 2.6.1 patch that I committed.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Attaching the 2.6.1 patch that I committed.

          People

          • Assignee:
            zxu zhihai xu
            Reporter:
            zxu zhihai xu
          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development