Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3619

ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ContainerMetrics is able to unregister itself during the getMetrics method, but that method can be called by MetricsSystemImpl.sampleMetrics which is trying to iterate the sources. This leads to a ConcurrentModificationException log like this:

      2015-05-11 14:00:20,360 [Timer for 'NodeManager' metrics system] WARN impl.MetricsSystemImpl: java.util.ConcurrentModificationException
      
      1. test.patch
        3 kB
        zhihai xu
      2. YARN-3619.000.patch
        14 kB
        zhihai xu
      3. YARN-3619.001.patch
        14 kB
        zhihai xu
      4. YARN-3619.alt.patch
        16 kB
        zhihai xu
      5. YARN-3619.branch-2.7.patch
        15 kB
        zhihai xu

        Issue Links

          Activity

          Hide
          jlowe Jason Lowe added a comment -

          This appears to have been caused by YARN-2984. Karthik Kambatla would you mind taking a look?

          Show
          jlowe Jason Lowe added a comment - This appears to have been caused by YARN-2984 . Karthik Kambatla would you mind taking a look?
          Hide
          kasha Karthik Kambatla added a comment -

          Zhihai pinged me offline mentioning he knows the root cause behind this. zhihai xu - assigning this to you.

          Show
          kasha Karthik Kambatla added a comment - Zhihai pinged me offline mentioning he knows the root cause behind this. zhihai xu - assigning this to you.
          Hide
          zxu zhihai xu added a comment -

          thanks Karthik Kambatla for assigning this JIRA to me.
          The root cause is exactly what Jason Lowe said. I just added a little more details based on Jason Lowe succinct comment.
          sampleMetrics will be called periodically in MetricsSystemImpl. sampleMetrics will iterate the sources in the following code:

              for (Entry<String, MetricsSourceAdapter> entry : sources.entrySet()) {
                if (sourceFilter == null || sourceFilter.accepts(entry.getKey())) {
                  snapshotMetrics(entry.getValue(), bufferBuilder);
                }
              }
          

          snapshotMetrics will be called to process every entry from sources
          The calling sequence which leads to a ConcurrentModificationException is
          snapshotMetrics => MetricsSourceAdapter#getMetrics => ContainerMetrics#getMetrics => MetricsSystemImpl#unregisterSource => sources.remove(name)
          the entry in the sources is removed when iterate the sources. So unregisterSource can't be called from getMetrics.

          I will prepare a patch for review.

          Show
          zxu zhihai xu added a comment - thanks Karthik Kambatla for assigning this JIRA to me. The root cause is exactly what Jason Lowe said. I just added a little more details based on Jason Lowe succinct comment. sampleMetrics will be called periodically in MetricsSystemImpl. sampleMetrics will iterate the sources in the following code: for (Entry< String , MetricsSourceAdapter> entry : sources.entrySet()) { if (sourceFilter == null || sourceFilter.accepts(entry.getKey())) { snapshotMetrics(entry.getValue(), bufferBuilder); } } snapshotMetrics will be called to process every entry from sources The calling sequence which leads to a ConcurrentModificationException is snapshotMetrics => MetricsSourceAdapter#getMetrics => ContainerMetrics#getMetrics => MetricsSystemImpl#unregisterSource => sources.remove(name) the entry in the sources is removed when iterate the sources . So unregisterSource can't be called from getMetrics. I will prepare a patch for review.
          Hide
          zxu zhihai xu added a comment -

          I attached a test patch which can reproduce this issue with the following stack trace:

          Running org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics
          Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.92 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics
          testContainerMetricsFinished(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics)  Time elapsed: 1.194 sec  <<< ERROR!
          java.util.ConcurrentModificationException: null
          	at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394)
          	at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:413)
          	at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:412)
          	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.sampleMetrics(MetricsSystemImpl.java:403)
          	at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFinished(TestContainerMetrics.java:144)
          Results :
          Tests in error: 
            TestContainerMetrics.testContainerMetricsFinished:144 » ConcurrentModification
          Tests run: 3, Failures: 0, Errors: 1, Skipped: 0
          
          Show
          zxu zhihai xu added a comment - I attached a test patch which can reproduce this issue with the following stack trace: Running org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.92 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics testContainerMetricsFinished(org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics) Time elapsed: 1.194 sec <<< ERROR! java.util.ConcurrentModificationException: null at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394) at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:413) at java.util.LinkedHashMap$EntryIterator.next(LinkedHashMap.java:412) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.sampleMetrics(MetricsSystemImpl.java:403) at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainerMetrics.testContainerMetricsFinished(TestContainerMetrics.java:144) Results : Tests in error: TestContainerMetrics.testContainerMetricsFinished:144 » ConcurrentModification Tests run: 3, Failures: 0, Errors: 1, Skipped: 0
          Hide
          zxu zhihai xu added a comment -

          I uploaded a patch YARN-3619.000.patch for review. I added a configuration NM_CONTAINER_METRICS_UNREGISTER_DELAY_MS to configure when to unregister the container metrics after it is finished. Because it may have potential memory leak If I schedule a thread to do unregistration at getMetrics.
          It looks like getMetrics will be called from two places:MetricsSystemImpl#sampleMetrics and MetricsSourceAdapter#getMBeanInfo.
          sampleMetrics won't be called if no sinks in MetricsSystemImpl. getMBeanInfo may not be called after registration if JMXJsonServlet#doGet is not called(no http Get request from JMX clients). It looks like there is a possibility that getMetrics won't be called after registration.

          Show
          zxu zhihai xu added a comment - I uploaded a patch YARN-3619 .000.patch for review. I added a configuration NM_CONTAINER_METRICS_UNREGISTER_DELAY_MS to configure when to unregister the container metrics after it is finished. Because it may have potential memory leak If I schedule a thread to do unregistration at getMetrics. It looks like getMetrics will be called from two places:MetricsSystemImpl#sampleMetrics and MetricsSourceAdapter#getMBeanInfo. sampleMetrics won't be called if no sinks in MetricsSystemImpl. getMBeanInfo may not be called after registration if JMXJsonServlet#doGet is not called(no http Get request from JMX clients). It looks like there is a possibility that getMetrics won't be called after registration.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 33s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 34s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 25m 14s Tests passed in hadoop-common.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 6m 41s Tests passed in hadoop-yarn-server-nodemanager.
              73m 17s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733882/YARN-3619.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c97f32e
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8003/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8003/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 34s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 25m 14s Tests passed in hadoop-common. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 6m 41s Tests passed in hadoop-yarn-server-nodemanager.     73m 17s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733882/YARN-3619.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c97f32e hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8003/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8003/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8003/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Moving out all non-critical / non-blocker issues that didn't make it out of 2.7.1 into 2.7.2.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Moving out all non-critical / non-blocker issues that didn't make it out of 2.7.1 into 2.7.2.
          Hide
          jlowe Jason Lowe added a comment -

          My apologies for the long delay, as this fell off my radar. The approach seems reasonable.

          The patch needs to be upmerged to trunk. In addition I'm wondering about the Timer handling. I think the Timer should be a daemon thread (we don't want to prolong NM shutdown due to this). Also it seems wasteful to dedicate a separate timer thread for every container that finished. It would be more efficient to share a timer that handles multiple timer tasks rather than spawn a thread for every timer task.

          Show
          jlowe Jason Lowe added a comment - My apologies for the long delay, as this fell off my radar. The approach seems reasonable. The patch needs to be upmerged to trunk. In addition I'm wondering about the Timer handling. I think the Timer should be a daemon thread (we don't want to prolong NM shutdown due to this). Also it seems wasteful to dedicate a separate timer thread for every container that finished. It would be more efficient to share a timer that handles multiple timer tasks rather than spawn a thread for every timer task.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12733882/YARN-3619.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6c17d31
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9311/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12733882/YARN-3619.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6c17d31 Console output https://builds.apache.org/job/PreCommit-YARN-Build/9311/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 31s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 59s There were no new javac warning messages.
          +1 javadoc 10m 14s There were no new javadoc warning messages.
          -1 release audit 0m 16s The applied patch generated 1 release audit warnings.
          -1 checkstyle 3m 0s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          -1 findbugs 6m 27s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 common tests 8m 22s Tests passed in hadoop-common.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 46s Tests passed in hadoop-yarn-server-nodemanager.
              71m 48s  



          Reason Tests
          FindBugs module:hadoop-yarn-server-nodemanager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764538/YARN-3619.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 5db371f
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9318/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9318/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 31s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 14s There were no new javadoc warning messages. -1 release audit 0m 16s The applied patch generated 1 release audit warnings. -1 checkstyle 3m 0s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. -1 findbugs 6m 27s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 common tests 8m 22s Tests passed in hadoop-common. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 1m 59s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 46s Tests passed in hadoop-yarn-server-nodemanager.     71m 48s   Reason Tests FindBugs module:hadoop-yarn-server-nodemanager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764538/YARN-3619.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 5db371f Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9318/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9318/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9318/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          Jason Lowe, thanks for the review! Yes, that is a very good suggestion. I uploaded a new patch YARN-3619.001.patch, which addressed all your comments(daemon thread and share a timer), please review it.

          Show
          zxu zhihai xu added a comment - Jason Lowe , thanks for the review! Yes, that is a very good suggestion. I uploaded a new patch YARN-3619 .001.patch, which addressed all your comments(daemon thread and share a timer), please review it.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for updating the patch! In the future, please do not remove patches unless absolutely necessary as it generates a lot of email churn and can lead to confusion as in this case. The patchbot comments above are not for the most recent patch applied despite the patch basename being the same. This subsequent patch should have simply been version 002.

          Latest patch looks good to me, checkstyle and release audit warnings were pre-existing. Kicking Jenkins again to comment on the latest patch.

          Show
          jlowe Jason Lowe added a comment - Thanks for updating the patch! In the future, please do not remove patches unless absolutely necessary as it generates a lot of email churn and can lead to confusion as in this case. The patchbot comments above are not for the most recent patch applied despite the patch basename being the same. This subsequent patch should have simply been version 002. Latest patch looks good to me, checkstyle and release audit warnings were pre-existing. Kicking Jenkins again to comment on the latest patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 20m 51s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 46s There were no new javac warning messages.
          +1 javadoc 10m 9s There were no new javadoc warning messages.
          -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
          -1 checkstyle 2m 58s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 32s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 6m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 29s Tests passed in hadoop-common.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 32s Tests passed in hadoop-yarn-server-nodemanager.
              69m 41s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764544/YARN-3619.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 195793c
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9323/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9323/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 51s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 10m 9s There were no new javadoc warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. -1 checkstyle 2m 58s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 6m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 29s Tests passed in hadoop-common. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 32s Tests passed in hadoop-yarn-server-nodemanager.     69m 41s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764544/YARN-3619.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 195793c Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9323/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9323/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9323/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          Thanks for the good suggestions Jason Lowe! Will do it next time to avoid confusion! Thanks for the thorough review!

          Show
          zxu zhihai xu added a comment - Thanks for the good suggestions Jason Lowe ! Will do it next time to avoid confusion! Thanks for the thorough review!
          Hide
          zxu zhihai xu added a comment -

          The checkstyle issues and release audit warnings for the latest patch YARN-3619.001.patch were pre-existing.

          Show
          zxu zhihai xu added a comment - The checkstyle issues and release audit warnings for the latest patch YARN-3619 .001.patch were pre-existing.
          Hide
          zxu zhihai xu added a comment -

          I just attached the previous confused patch(YARN-3619.alt.patch) which I removed for comparison. It shares both timer and timer task, which is more complicated than YARN-3619.001.patch, So I think YARN-3619.001.patch(share timer only) is a better approach, which is simpler and more accurately controls the time to unregister the container metrics.

          Show
          zxu zhihai xu added a comment - I just attached the previous confused patch( YARN-3619 .alt.patch) which I removed for comparison. It shares both timer and timer task, which is more complicated than YARN-3619 .001.patch, So I think YARN-3619 .001.patch(share timer only) is a better approach, which is simpler and more accurately controls the time to unregister the container metrics.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 16s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 50s There were no new javac warning messages.
          +1 javadoc 10m 5s There were no new javadoc warning messages.
          -1 release audit 0m 16s The applied patch generated 1 release audit warnings.
          -1 checkstyle 2m 59s The applied patch generated 1 new checkstyle issues (total was 211, now 211).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          -1 findbugs 6m 17s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          -1 common tests 6m 32s Tests failed in hadoop-common.
          +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api.
          +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 41s Tests passed in hadoop-yarn-server-nodemanager.
              69m 6s  



          Reason Tests
          FindBugs module:hadoop-yarn-server-nodemanager
          Failed unit tests hadoop.fs.sftp.TestSFTPFileSystem



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764776/YARN-3619.alt.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 439f43a
          Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9330/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9330/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 16s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 10m 5s There were no new javadoc warning messages. -1 release audit 0m 16s The applied patch generated 1 release audit warnings. -1 checkstyle 2m 59s The applied patch generated 1 new checkstyle issues (total was 211, now 211). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. -1 findbugs 6m 17s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. -1 common tests 6m 32s Tests failed in hadoop-common. +1 yarn tests 0m 23s Tests passed in hadoop-yarn-api. +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 41s Tests passed in hadoop-yarn-server-nodemanager.     69m 6s   Reason Tests FindBugs module:hadoop-yarn-server-nodemanager Failed unit tests hadoop.fs.sftp.TestSFTPFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764776/YARN-3619.alt.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 439f43a Release Audit https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html hadoop-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9330/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9330/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9330/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1 for the .001 patch. The patch doesn't apply to branch-2.7 cleanly, and I'd like to get it fixed in 2.7.2. Can you provide a patch against branch-2.7?

          Show
          jlowe Jason Lowe added a comment - +1 for the .001 patch. The patch doesn't apply to branch-2.7 cleanly, and I'd like to get it fixed in 2.7.2. Can you provide a patch against branch-2.7?
          Hide
          zxu zhihai xu added a comment -

          thanks Jason Lowe! Yes, I uploaded a patch YARN-3619.branch-2.7.patch based on branch-2.7.

          Show
          zxu zhihai xu added a comment - thanks Jason Lowe ! Yes, I uploaded a patch YARN-3619 .branch-2.7.patch based on branch-2.7.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764832/YARN-3619.branch-2.7.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision branch-2 / 7964b13
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9335/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764832/YARN-3619.branch-2.7.patch Optional Tests javadoc javac unit findbugs checkstyle git revision branch-2 / 7964b13 Console output https://builds.apache.org/job/PreCommit-YARN-Build/9335/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1 for the branch-2.7 patch. Committing this.

          Show
          jlowe Jason Lowe added a comment - +1 for the branch-2.7 patch. Committing this.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks, Zhihai! I committed this to trunk, branch-2, and branch-2.7

          Show
          jlowe Jason Lowe added a comment - Thanks, Zhihai! I committed this to trunk, branch-2, and branch-2.7
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8559 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8559/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8559 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8559/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #481 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/481/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #481 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/481/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          Hide
          zxu zhihai xu added a comment -

          Thanks Jason Lowe for reviewing and committing the very old patch! I really appreciate it.

          Show
          zxu zhihai xu added a comment - Thanks Jason Lowe for reviewing and committing the very old patch! I really appreciate it.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #473 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/473/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #473 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/473/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1212/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1212/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2417 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2417/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2417 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2417/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #447 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/447/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #447 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/447/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2387 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2387/)
          YARN-3619. ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2387 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2387/ ) YARN-3619 . ContainerMetrics unregisters during getMetrics and leads to (jlowe: rev fdf02d1f26cea372bf69e071f57b8bfc09c092c4) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java hadoop-yarn-project/CHANGES.txt

            People

            • Assignee:
              zxu zhihai xu
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development