Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11361

Fix a race condition in MetricsSourceAdapter.updateJmxCache

    Details

      Description

      Caused by: java.lang.NullPointerException
      	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateAttrCache(MetricsSourceAdapter.java:247)
      	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:177)
      	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getAttribute(MetricsSourceAdapter.java:102)
      	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
      
      1. HDFS-7487.patch
        0.7 kB
        Brahma Reddy Battula
      2. HADOOP-11361.patch
        0.7 kB
        Brahma Reddy Battula
      3. HADOOP-11361-002.patch
        0.9 kB
        Brahma Reddy Battula
      4. HADOOP-111361-003.patch
        2 kB
        Brahma Reddy Battula
      5. HADOOP-11361-004.patch
        0.9 kB
        Brahma Reddy Battula
      6. HADOOP-11361-005.patch
        4 kB
        Vinayakumar B
      7. HADOOP-11361-005.patch
        6 kB
        Vinayakumar B
      8. HADOOP-11361-006.patch
        4 kB
        Vinayakumar B
      9. HADOOP-11361-007.patch
        4 kB
        Vinayakumar B
      10. HADOOP-11361.008.patch
        4 kB
        Tsuyoshi Ozawa
      11. HADOOP-11361-009.patch
        3 kB
        Vinayakumar B

        Issue Links

          Activity

          Hide
          sjlee0 Sangjin Lee added a comment -

          Cherry-picked it to 2.6.5. Picked also HADOOP-11301, HADOOP-12348, and HADOOP-12482 before this.

          Show
          sjlee0 Sangjin Lee added a comment - Cherry-picked it to 2.6.5. Picked also HADOOP-11301 , HADOOP-12348 , and HADOOP-12482 before this.
          Hide
          ctrezzo Chris Trezzo added a comment -

          Tsuyoshi Ozawa I noticed that the target version for this jira states 2.6.5, but the fix version does not list 2.6.5 (in-line with the commit not being in branch-2.6). Do we want to backport to branch-2.6 as well, or should we remove 2.6.5 from target versions? Thanks!

          Show
          ctrezzo Chris Trezzo added a comment - Tsuyoshi Ozawa I noticed that the target version for this jira states 2.6.5, but the fix version does not list 2.6.5 (in-line with the commit not being in branch-2.6). Do we want to backport to branch-2.6 as well, or should we remove 2.6.5 from target versions? Thanks!
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks All.

          Show
          vinayrpet Vinayakumar B added a comment - Thanks All.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Many thanks to you all Tsuyoshi Ozawa, Vinayakumar B, Brahma Reddy Battula, and Jason Lowe, nice collaborative work here!

          Show
          yzhangal Yongjun Zhang added a comment - Many thanks to you all Tsuyoshi Ozawa , Vinayakumar B , Brahma Reddy Battula , and Jason Lowe , nice collaborative work here!
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk and branch-2, branch-2.8, branch-2.7. Thanks Vinayakumar B, Yongjun Zhang, and Brahma Reddy Battula for the iterative updating. Thanks for Jason Lowe for reverting previous patch.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2, branch-2.8, branch-2.7. Thanks Vinayakumar B , Yongjun Zhang , and Brahma Reddy Battula for the iterative updating. Thanks for Jason Lowe for reverting previous patch.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10088 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10088/)
          HADOOP-11361. Fix a race condition in (ozawa: rev 77ffe7621212be9f462ca37a542a13d167eca4e0)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10088 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10088/ ) HADOOP-11361 . Fix a race condition in (ozawa: rev 77ffe7621212be9f462ca37a542a13d167eca4e0) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          +1, checking this in.

          Show
          ozawa Tsuyoshi Ozawa added a comment - +1, checking this in.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 8m 53s trunk passed
          +1 compile 9m 33s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 14s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 0m 52s trunk passed
          +1 mvninstall 0m 50s the patch passed
          +1 compile 9m 25s the patch passed
          +1 javac 9m 25s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 39s the patch passed
          +1 javadoc 0m 49s the patch passed
          -1 unit 8m 26s hadoop-common in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          48m 12s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817627/HADOOP-11361.008.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 300a01ee7a18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 438b7c5
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 53s trunk passed +1 compile 9m 33s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 14s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 0m 52s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 9m 25s the patch passed +1 javac 9m 25s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 39s the patch passed +1 javadoc 0m 49s the patch passed -1 unit 8m 26s hadoop-common in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 48m 12s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817627/HADOOP-11361.008.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 300a01ee7a18 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 438b7c5 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9980/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 1s trunk passed
          +1 compile 6m 53s trunk passed
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 16s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 36s the patch passed
          +1 compile 6m 47s the patch passed
          +1 javac 6m 47s the patch passed
          +1 checkstyle 0m 22s the patch passed
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 27s the patch passed
          +1 javadoc 0m 45s the patch passed
          +1 unit 7m 9s hadoop-common in the patch passed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          37m 38s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817636/HADOOP-11361-009.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e983d40cdfd5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 438b7c5
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9983/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9983/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 1s trunk passed +1 compile 6m 53s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 16s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 36s the patch passed +1 compile 6m 47s the patch passed +1 javac 6m 47s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 27s the patch passed +1 javadoc 0m 45s the patch passed +1 unit 7m 9s hadoop-common in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 37m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817636/HADOOP-11361-009.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e983d40cdfd5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 438b7c5 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9983/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9983/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Attached the patch with lastRecs changed to Iterable and getMetrics() return Signature changed to Iterable as well.

          Show
          vinayrpet Vinayakumar B added a comment - Attached the patch with lastRecs changed to Iterable and getMetrics() return Signature changed to Iterable as well.
          Hide
          vinayrpet Vinayakumar B added a comment - - edited

          I found that the change of the signature, (in concrete, return type, of (getMetrics(builder, all)) breaks other tests - it can cause test failures of TestContainerMetrics because of type cast failures, although Jenkins could not find it.

          Can you please point me the failure place.? Since Iterable is a interface, and List also implements it, I dont see any problem in current signature. And returned builder.getRecords() is also a List itself.

          To fix the problem, I attach v8 patch without changing signature(return type) of getMetrics(builder, all), and add private List<MetricsRecordImpl> getMetrics(builder) as a wrapper for private call in MetricsSourceAdapter. This hack also preserves backward compatibility.

          Instead of this Hack, we can change the lastRecs itself to Iterable. Even pre-patch declaration of lastRecs was Iterable itself.

          Show
          vinayrpet Vinayakumar B added a comment - - edited I found that the change of the signature, (in concrete, return type, of (getMetrics(builder, all)) breaks other tests - it can cause test failures of TestContainerMetrics because of type cast failures, although Jenkins could not find it. Can you please point me the failure place.? Since Iterable is a interface, and List also implements it, I dont see any problem in current signature. And returned builder.getRecords() is also a List itself. To fix the problem, I attach v8 patch without changing signature(return type) of getMetrics(builder, all), and add private List<MetricsRecordImpl> getMetrics(builder) as a wrapper for private call in MetricsSourceAdapter. This hack also preserves backward compatibility. Instead of this Hack, we can change the lastRecs itself to Iterable . Even pre-patch declaration of lastRecs was Iterable itself.
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Vinayakumar B Yongjun Zhang I found that the change of the signature, (in concrete, return type, of (getMetrics(builder, all)) breaks other tests - it can cause test failures of TestContainerMetrics because of type cast failures, although Jenkins could not find it.

          To fix the problem, I attach v8 patch without changing signature(return type) of getMetrics(builder, all), and add private List<MetricsRecordImpl> getMetrics(builder) as a wrapper for private call in MetricsSourceAdapter. This hack also preserves backward compatibility.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Vinayakumar B Yongjun Zhang I found that the change of the signature, (in concrete, return type, of ( getMetrics(builder, all) ) breaks other tests - it can cause test failures of TestContainerMetrics because of type cast failures, although Jenkins could not find it. To fix the problem, I attach v8 patch without changing signature(return type) of getMetrics(builder, all) , and add private List<MetricsRecordImpl> getMetrics(builder) as a wrapper for private call in MetricsSourceAdapter. This hack also preserves backward compatibility.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 20s trunk passed
          +1 compile 7m 8s trunk passed
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 23s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 38s the patch passed
          +1 compile 7m 2s the patch passed
          +1 javac 7m 2s the patch passed
          +1 checkstyle 0m 22s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 1s The patch has no whitespace issues.
          +1 findbugs 1m 28s the patch passed
          +1 javadoc 0m 47s the patch passed
          +1 unit 8m 6s hadoop-common in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          39m 36s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817598/HADOOP-11361-007.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1795a60841d0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 06c56ff
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9977/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9977/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 20s trunk passed +1 compile 7m 8s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 23s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 7m 2s the patch passed +1 javac 7m 2s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 47s the patch passed +1 unit 8m 6s hadoop-common in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 39m 36s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817598/HADOOP-11361-007.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1795a60841d0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 06c56ff Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9977/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9977/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Tsuyoshi Ozawa, thanks for your review, I'm +1 on rev 007, wonder if it looks good to you. Hope we can get it in tomorrow if everything looks good. Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Tsuyoshi Ozawa , thanks for your review, I'm +1 on rev 007, wonder if it looks good to you. Hope we can get it in tomorrow if everything looks good. Thanks.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Vinayakumar B for the new revs, +1 on 007 pending jenkins test.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Vinayakumar B for the new revs, +1 on 007 pending jenkins test.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Updated the patch. Removed redundant check.

          Show
          vinayrpet Vinayakumar B added a comment - Updated the patch. Removed redundant check.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks guys, that's another improvement!

          One little thing:

              synchronized (this) {
                if (lastRecs != null) {
                  updateAttrCache(lastRecs);
                  if (getAllMetrics) {  <====== this check is redundant
                    updateInfoCache(lastRecs);
                  }
                }
                jmxCacheTS = Time.now();
                lastRecsCleared = true;
              }
          

          The if (getAllMetrics) in the above code is redundant, because it has to be true when lastRecs is not null.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks guys, that's another improvement! One little thing: synchronized ( this ) { if (lastRecs != null ) { updateAttrCache(lastRecs); if (getAllMetrics) { <====== this check is redundant updateInfoCache(lastRecs); } } jmxCacheTS = Time.now(); lastRecsCleared = true ; } The if (getAllMetrics) in the above code is redundant, because it has to be true when lastRecs is not null. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 13s trunk passed
          +1 compile 7m 18s trunk passed
          +1 checkstyle 0m 23s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 23s trunk passed
          +1 javadoc 0m 46s trunk passed
          +1 mvninstall 0m 39s the patch passed
          +1 compile 7m 13s the patch passed
          +1 javac 7m 13s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 30s the patch passed
          +1 javadoc 0m 46s the patch passed
          -1 unit 17m 0s hadoop-common in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          48m 58s



          Reason Tests
          Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817579/HADOOP-11361-006.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e7eadb688025 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 06c56ff
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 13s trunk passed +1 compile 7m 18s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 23s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 39s the patch passed +1 compile 7m 13s the patch passed +1 javac 7m 13s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 30s the patch passed +1 javadoc 0m 46s the patch passed -1 unit 17m 0s hadoop-common in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 48m 58s Reason Tests Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817579/HADOOP-11361-006.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e7eadb688025 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 06c56ff Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9974/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Updated the patch according to comments.
          Retained the original parameters for getMetrics() to avoid breakage in callers, if any, implemented outside Hadoop Code.

          Show
          vinayrpet Vinayakumar B added a comment - Updated the patch according to comments. Retained the original parameters for getMetrics() to avoid breakage in callers, if any, implemented outside Hadoop Code.
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          3. How about adding a comment why we release lock before calling private method getMetrics in updateJmxCache?

              // HADOOP-11361: Release lock here for avoid deadlock between
              // MetricsSystemImpl's lock and MetricsSourceAdapter's lock. 
              List<MetricsRecordImpl> lastRecs = null;
              if (getAllMetrics) {
                lastRecs = getMetrics(new MetricsCollectorImpl());
              }
          
          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited 3. How about adding a comment why we release lock before calling private method getMetrics in updateJmxCache ? // HADOOP-11361: Release lock here for avoid deadlock between // MetricsSystemImpl's lock and MetricsSourceAdapter's lock. List<MetricsRecordImpl> lastRecs = null ; if (getAllMetrics) { lastRecs = getMetrics( new MetricsCollectorImpl()); }
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Thanks Vinayakumar B and Yongjun Zhang for your reviews. My comments are as follows:

          1. This is related to Youngjun's comment about null check, and I prefer to remove the variable MetricsCollectorImpl builder and to use local variable List<MetricsRecordImpl> lastRecs for simplicity:

              List<MetricsRecordImpl> lastRecs = null;
              if (getAllMetrics) {
                lastRecs = getMetrics(new MetricsCollectorImpl());
              }
          
              synchronized(this) {
                if (lastRecs != null) {
                  updateAttrCache(lastRecs);
                  if (getAllMetrics) {
                    updateInfoCache(lastRecs);
                  }
                }
                jmxCacheTS = Time.now();
                lastRecsCleared = true;
              }
          

          In this case, we also need to update getMetrics(MetricsCollectorImpl builder) to return List<MetricsRecordImpl> instead of Iterable<MetricsRecordImpl>.

          2. This is minor nits comment, but I think we should add null-check assertions against lastRecs in following methods: private void updateInfoCache(List<MetricsRecordImpl> lastRecs), private int updateAttrCache(List<MetricsRecordImpl> lastRecs). It increases readability and simplicity. On trunk, we can use Nonnull annotation(https://blogs.oracle.com/java-platform-group/entry/java_8_s_new_type), but it's been introduced since jdk8. Assert.checkNotNull provided in Guava is also enough.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Thanks Vinayakumar B and Yongjun Zhang for your reviews. My comments are as follows: 1. This is related to Youngjun's comment about null check, and I prefer to remove the variable MetricsCollectorImpl builder and to use local variable List<MetricsRecordImpl> lastRecs for simplicity: List<MetricsRecordImpl> lastRecs = null ; if (getAllMetrics) { lastRecs = getMetrics( new MetricsCollectorImpl()); } synchronized ( this ) { if (lastRecs != null ) { updateAttrCache(lastRecs); if (getAllMetrics) { updateInfoCache(lastRecs); } } jmxCacheTS = Time.now(); lastRecsCleared = true ; } In this case, we also need to update getMetrics(MetricsCollectorImpl builder) to return List<MetricsRecordImpl> instead of Iterable<MetricsRecordImpl> . 2. This is minor nits comment, but I think we should add null-check assertions against lastRecs in following methods: private void updateInfoCache(List<MetricsRecordImpl> lastRecs) , private int updateAttrCache(List<MetricsRecordImpl> lastRecs) . It increases readability and simplicity. On trunk, we can use Nonnull annotation( https://blogs.oracle.com/java-platform-group/entry/java_8_s_new_type ), but it's been introduced since jdk8. Assert.checkNotNull provided in Guava is also enough.
          Hide
          yzhangal Yongjun Zhang added a comment - - edited

          Thanks Tsuyoshi Ozawa and Vinayakumar B for the comments and Vinay for the updated patch!

          The latest patch looks good to me, except for two little things:
          1. The builder could be NULL here

             synchronized(this) {
                List<MetricsRecordImpl> lastRecs = builder.getRecords();
                updateAttrCache(lastRecs);
                if (getAllMetrics) {
                  updateInfoCache(lastRecs);
                }
          

          The calls need to be conditioned by getAllMetrics.

          2. The revision name should be 006 to avoid confusion.

          Thanks a lot.

          Show
          yzhangal Yongjun Zhang added a comment - - edited Thanks Tsuyoshi Ozawa and Vinayakumar B for the comments and Vinay for the updated patch! The latest patch looks good to me, except for two little things: 1. The builder could be NULL here synchronized ( this ) { List<MetricsRecordImpl> lastRecs = builder.getRecords(); updateAttrCache(lastRecs); if (getAllMetrics) { updateInfoCache(lastRecs); } The calls need to be conditioned by getAllMetrics . 2. The revision name should be 006 to avoid confusion. Thanks a lot.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 41s trunk passed
          +1 compile 8m 44s trunk passed
          +1 checkstyle 0m 23s trunk passed
          +1 mvnsite 1m 3s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 38s trunk passed
          +1 javadoc 0m 52s trunk passed
          +1 mvninstall 0m 42s the patch passed
          +1 compile 8m 20s the patch passed
          +1 javac 8m 20s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 48s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 49s the patch passed
          +1 unit 8m 22s hadoop-common in the patch passed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          45m 30s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            Possible null pointer dereference of builder in org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache() Dereferenced at MetricsSourceAdapter.java:builder in org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache() Dereferenced at MetricsSourceAdapter.java:[line 186]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817476/HADOOP-11361-005.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3625f50cbd4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7705812
          Default Java 1.8.0_91
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 41s trunk passed +1 compile 8m 44s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 52s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 8m 20s the patch passed +1 javac 8m 20s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 48s hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 49s the patch passed +1 unit 8m 22s hadoop-common in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 45m 30s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Possible null pointer dereference of builder in org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache() Dereferenced at MetricsSourceAdapter.java:builder in org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache() Dereferenced at MetricsSourceAdapter.java: [line 186] Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817476/HADOOP-11361-005.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3625f50cbd4b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7705812 Default Java 1.8.0_91 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9971/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Fixed compilation errors

          Show
          vinayrpet Vinayakumar B added a comment - Fixed compilation errors
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 24s trunk passed
          +1 compile 8m 26s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 5s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 34s trunk passed
          +1 javadoc 0m 51s trunk passed
          -1 mvninstall 0m 39s hadoop-common in the patch failed.
          -1 compile 0m 59s root in the patch failed.
          -1 javac 0m 59s root in the patch failed.
          +1 checkstyle 0m 23s the patch passed
          -1 mvnsite 0m 41s hadoop-common in the patch failed.
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 26s hadoop-common in the patch failed.
          +1 javadoc 0m 48s the patch passed
          -1 unit 0m 45s hadoop-common in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          28m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817468/HADOOP-11361-005.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c3fa1317534a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7705812
          Default Java 1.8.0_91
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 24s trunk passed +1 compile 8m 26s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 34s trunk passed +1 javadoc 0m 51s trunk passed -1 mvninstall 0m 39s hadoop-common in the patch failed. -1 compile 0m 59s root in the patch failed. -1 javac 0m 59s root in the patch failed. +1 checkstyle 0m 23s the patch passed -1 mvnsite 0m 41s hadoop-common in the patch failed. +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 26s hadoop-common in the patch failed. +1 javadoc 0m 48s the patch passed -1 unit 0m 45s hadoop-common in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 28m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817468/HADOOP-11361-005.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c3fa1317534a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7705812 Default Java 1.8.0_91 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9970/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          It looks to me that the acquirement of lastRecs and the updateAttrCache should be protected in a same synchronized(this) block, to avoid this race condition

          In that case, there is no need to have a field lastRecs.
          lastRecs was used only in updateAttrCache() and updateInfoCache().
          We can have the method local lastRecs and same can be passed to updateAttrCache() and updateInfoCache().
          getMetrics() can return just builder.getRecords().
          So total number of synchronized calls will be reduced.

          Attaching the proposed changes.

          Show
          vinayrpet Vinayakumar B added a comment - It looks to me that the acquirement of lastRecs and the updateAttrCache should be protected in a same synchronized(this) block, to avoid this race condition In that case, there is no need to have a field lastRecs . lastRecs was used only in updateAttrCache() and updateInfoCache() . We can have the method local lastRecs and same can be passed to updateAttrCache() and updateInfoCache() . getMetrics() can return just builder.getRecords() . So total number of synchronized calls will be reduced. Attaching the proposed changes.
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Yongjun Zhang +1 for the idea. It avoid the deadlock between MetricsSystemImpl's lock and MetricsSourceAdapter's lock

          getMetricsPart1 should be named as getMetrics(), and getMetricsPart2 should be renamed as updateAndGetLastRecs, since it's more straightforward. Could you update the patch?

          Brahma Reddy Battula Vinayakumar B could you review Yongjun Zhang's patch if you have time? I would like to double-check new patch if possible.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Yongjun Zhang +1 for the idea. It avoid the deadlock between MetricsSystemImpl's lock and MetricsSourceAdapter's lock getMetricsPart1 should be named as getMetrics (), and getMetricsPart2 should be renamed as updateAndGetLastRecs , since it's more straightforward. Could you update the patch? Brahma Reddy Battula Vinayakumar B could you review Yongjun Zhang 's patch if you have time? I would like to double-check new patch if possible.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Brahma Reddy Battula, Vinayakumar B, Tsuyoshi Ozawa,

          Thanks for your earlier work/discussion here. Would you please help commenting on my last comment? Thanks.

          --Yongjun

          Show
          yzhangal Yongjun Zhang added a comment - Hi Brahma Reddy Battula , Vinayakumar B , Tsuyoshi Ozawa , Thanks for your earlier work/discussion here. Would you please help commenting on my last comment? Thanks. --Yongjun
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Vinayakumar B and Tsuyoshi Ozawa,

          Thanks for the discussion.

          First of all, I tend to agree with Tsuyoshi on the AND/OR analysis, however, I saw that all calls to getMetrics method passed true to all parameter, so this change won't impact the current final result. But we certainly can address this issue, either as a separate jira, or a side effect of the fix here.

          Thanks Vinayakumar B for the explanation of the race condition. Sorry I did miss the "return" statement when making the earlier comment.

          It looks to me that the acquirement of lastRecs and the updateAttrCache should be protected in a same synchronized(this) block, to avoid this race condition. And indeed the two threads Vinay described intend to have their own builder, their own lastRecs. Say, we can break down the getMetrics method into two parts,

            void getMetricsPart1(MetricsCollectorImpl builder,
                                                   boolean all) {
              builder.setRecordFilter(recordFilter).setMetricFilter(metricFilter);
              synchronized(this) {
                if (lastRecs == null || jmxCacheTS == 0) {
                  all = true; // Get all the metrics to populate the sink caches
                }
              }
              try {
                source.getMetrics(builder, all);
              } catch (Exception e) {
                LOG.error("Error getting metrics from source "+ name, e);
              }
              for (MetricsRecordBuilderImpl rb : builder) {
                for (MetricsTag t : injectedTags) {
                  rb.add(t);
                }
              }
            }
          
            Iterable<MetricsRecordImpl> getMetricsPart2(MetricsCollectorImpl builder) {
                lastRecs = builder.getRecords();
                return lastRecs;
            }
          
          

          and change

              if (getAllMetrics) {
                MetricsCollectorImpl builder = new MetricsCollectorImpl();
                getMetrics(builder, true);
              }
          
              synchronized(this) {
                updateAttrCache();
                if (getAllMetrics) {
                  updateInfoCache();
                }
                jmxCacheTS = Time.now();
                lastRecs = null;  // in case regular interval update is not running
                lastRecsCleared = true;
              }
          
          

          to

              MetricsCollectorImpl builder = NULL;
              if (getAllMetrics) {
                builder = new MetricsCollectorImpl();
                getMetricsPart1(builder, true);
              }
          
              synchronized(this) {
                if (getAllMetrics) {
                   getMetricsPart2(builder);
                }
                updateAttrCache();
                if (getAllMetrics) {
                  updateInfoCache();
                }
                jmxCacheTS = Time.now();
                lastRecs = null;  // in case regular interval update is not running
                lastRecsCleared = true;
              }
          

          This is just to throw an idea to see what's the best solution. The method names can be adjusted.

          What do you guys think?

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Vinayakumar B and Tsuyoshi Ozawa , Thanks for the discussion. First of all, I tend to agree with Tsuyoshi on the AND/OR analysis, however, I saw that all calls to getMetrics method passed true to all parameter, so this change won't impact the current final result. But we certainly can address this issue, either as a separate jira, or a side effect of the fix here. Thanks Vinayakumar B for the explanation of the race condition. Sorry I did miss the "return" statement when making the earlier comment. It looks to me that the acquirement of lastRecs and the updateAttrCache should be protected in a same synchronized(this) block, to avoid this race condition. And indeed the two threads Vinay described intend to have their own builder, their own lastRecs. Say, we can break down the getMetrics method into two parts, void getMetricsPart1(MetricsCollectorImpl builder, boolean all) { builder.setRecordFilter(recordFilter).setMetricFilter(metricFilter); synchronized ( this ) { if (lastRecs == null || jmxCacheTS == 0) { all = true ; // Get all the metrics to populate the sink caches } } try { source.getMetrics(builder, all); } catch (Exception e) { LOG.error( "Error getting metrics from source " + name, e); } for (MetricsRecordBuilderImpl rb : builder) { for (MetricsTag t : injectedTags) { rb.add(t); } } } Iterable<MetricsRecordImpl> getMetricsPart2(MetricsCollectorImpl builder) { lastRecs = builder.getRecords(); return lastRecs; } and change if (getAllMetrics) { MetricsCollectorImpl builder = new MetricsCollectorImpl(); getMetrics(builder, true ); } synchronized ( this ) { updateAttrCache(); if (getAllMetrics) { updateInfoCache(); } jmxCacheTS = Time.now(); lastRecs = null ; // in case regular interval update is not running lastRecsCleared = true ; } to MetricsCollectorImpl builder = NULL; if (getAllMetrics) { builder = new MetricsCollectorImpl(); getMetricsPart1(builder, true ); } synchronized ( this ) { if (getAllMetrics) { getMetricsPart2(builder); } updateAttrCache(); if (getAllMetrics) { updateInfoCache(); } jmxCacheTS = Time.now(); lastRecs = null ; // in case regular interval update is not running lastRecsCleared = true ; } This is just to throw an idea to see what's the best solution. The method names can be adjusted. What do you guys think? Thanks.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Thank you for explanation and reviews.I took a look more deeper. I think we should check overall semantics of MetricsSourceAdapter instead of doing workaround.

          At first, I suspect that getMetrics has a semantics bug. A following condition check whether infoCache should be updated.

                if (lastRecs == null && jmxCacheTS == 0) {
                        all = true; // Get all the metrics to populate the sink caches
                }
          

          infoCache should be updated in following cases:
          1. After updateAttrCache is called. It is expressed as lastRecs is null.
          2. Before initialization is done - before calling updateJmxCache. It is expressed as jmxCacheTS == 0.

          I think these condition should be connected with OR not AND, so it can be fixed as follows:

                if (lastRecs == null || jmxCacheTS == 0) {
                  all = true; // Get all the metrics to populate the sink caches
                }
          

          What do you think?

          Next, the NPE related problem:

          Race condition is there between two threads calling updateJmxCache() at same time.

          You're right. v3 patch fixed the race condition, but it introduced deadlock between JMXJsonServlet and ResourceManager's MetricSystem as Jason mentioned on HADOOP-12594:

          The timer thread has the MetricsSystemImpl lock and is trying to grab the MetricsSourceAdapter lock. In the meantime the JMX thread has the MetricsSourceAdapter lock and is trying to grab the MetricsSystemImpl lock. The locking order isn't consistent so we deadlocked.

          Brahma's solution is a bit tricky, so please let me confirm for a while.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Thank you for explanation and reviews.I took a look more deeper. I think we should check overall semantics of MetricsSourceAdapter instead of doing workaround. At first, I suspect that getMetrics has a semantics bug. A following condition check whether infoCache should be updated. if (lastRecs == null && jmxCacheTS == 0) { all = true ; // Get all the metrics to populate the sink caches } infoCache should be updated in following cases: 1. After updateAttrCache is called. It is expressed as lastRecs is null. 2. Before initialization is done - before calling updateJmxCache . It is expressed as jmxCacheTS == 0 . I think these condition should be connected with OR not AND , so it can be fixed as follows: if (lastRecs == null || jmxCacheTS == 0) { all = true ; // Get all the metrics to populate the sink caches } What do you think? Next, the NPE related problem: Race condition is there between two threads calling updateJmxCache() at same time. You're right. v3 patch fixed the race condition, but it introduced deadlock between JMXJsonServlet and ResourceManager's MetricSystem as Jason mentioned on HADOOP-12594 : The timer thread has the MetricsSystemImpl lock and is trying to grab the MetricsSourceAdapter lock. In the meantime the JMX thread has the MetricsSourceAdapter lock and is trying to grab the MetricsSystemImpl lock. The locking order isn't consistent so we deadlocked. Brahma's solution is a bit tricky, so please let me confirm for a while.
          Hide
          vinayrpet Vinayakumar B added a comment -

          If one thread enters the above method with Time.now() - jmxCacheTS >= jmxCacheTTL being false, then getAllMetrics will continue to be false, then when updateAttrCache() is called, it will hit NULL lastRecs

          From your earlier comment linked, if Time.now() - jmxCacheTS >= jmxCacheTTL is false, doesnt it return from there itself?

          It looks to me that it's actually not a race condition issue (otherwise please would you please explain what race condition you are seeing),

          Race condition is there between two threads calling updateJmxCache() at same time.

              if (getAllMetrics) {
                MetricsCollectorImpl builder = new MetricsCollectorImpl();
                getMetrics(builder, true);
              }
          
              synchronized(this) {
                updateAttrCache();
                if (getAllMetrics) {
                  updateInfoCache();
                }
                jmxCacheTS = Time.now();
                lastRecs = null;  // in case regular interval update is not running
                lastRecsCleared = true;
              }

          1. Consider both threads t1 and t2 waiting at the entrance of synchronized(this) {, i.e. line no : 183.
          2. t1 gets inside and makes the lastRecs = null and lastRecsCleared=true
          3. now since t2 already passed the line of reloading the lastRecs in getMetrics(..) line, t2 will get NPE in updateAttrCache().

          Given that the code is doing lastRecsCleared = true;, the comment "Not making LastRecs null ..." is a bit contradictory.

          To make it less contradictory, how about renaming lastRecsCleared to something, reloadRecs?

          . So seems to me that the solution would be to fix MetricsSourceAdapter#updateAttrCache to let it check whether lastRecs is NULL, and not to access it if so.

          This also works, since the lastRecs is used only to update attrCache and infoCache, avoiding updating these in case of lastRecs == null will not be a problem.

          Show
          vinayrpet Vinayakumar B added a comment - If one thread enters the above method with Time.now() - jmxCacheTS >= jmxCacheTTL being false, then getAllMetrics will continue to be false, then when updateAttrCache() is called, it will hit NULL lastRecs From your earlier comment linked, if Time.now() - jmxCacheTS >= jmxCacheTTL is false, doesnt it return from there itself? It looks to me that it's actually not a race condition issue (otherwise please would you please explain what race condition you are seeing), Race condition is there between two threads calling updateJmxCache() at same time. if (getAllMetrics) { MetricsCollectorImpl builder = new MetricsCollectorImpl(); getMetrics(builder, true ); } synchronized ( this ) { updateAttrCache(); if (getAllMetrics) { updateInfoCache(); } jmxCacheTS = Time.now(); lastRecs = null ; // in case regular interval update is not running lastRecsCleared = true ; } 1. Consider both threads t1 and t2 waiting at the entrance of synchronized(this) { , i.e. line no : 183. 2. t1 gets inside and makes the lastRecs = null and lastRecsCleared=true 3. now since t2 already passed the line of reloading the lastRecs in getMetrics(..) line, t2 will get NPE in updateAttrCache() . Given that the code is doing lastRecsCleared = true;, the comment "Not making LastRecs null ..." is a bit contradictory. To make it less contradictory, how about renaming lastRecsCleared to something, reloadRecs ? . So seems to me that the solution would be to fix MetricsSourceAdapter#updateAttrCache to let it check whether lastRecs is NULL, and not to access it if so. This also works, since the lastRecs is used only to update attrCache and infoCache , avoiding updating these in case of lastRecs == null will not be a problem.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Brahma Reddy Battula,

          I took a quick look at your latest patch,

          You removed

          lastRecs = null;  // in case regular interval update is not running
          

          and added

          189	      // Not making LastRecs null due to Race between multiple readers and
          190	      // updater
          190	      lastRecsCleared = true;
          

          Given that the code is doing lastRecsCleared = true;, the comment "Not making LastRecs null ..." is a bit contradictory.

          It looks to me that it's actually not a race condition issue (otherwise please would you please explain what race condition you are seeing), because it looks that a single thread would run into the same problem as explained in my comment linked above. So seems to me that the solution would be to fix MetricsSourceAdapter#updateAttrCache to let it check whether lastRecs is NULL, and not to access it if so.

          Does it make sense to you?

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Brahma Reddy Battula , I took a quick look at your latest patch, You removed lastRecs = null ; // in case regular interval update is not running and added 189 // Not making LastRecs null due to Race between multiple readers and 190 // updater 190 lastRecsCleared = true ; Given that the code is doing lastRecsCleared = true; , the comment "Not making LastRecs null ..." is a bit contradictory. It looks to me that it's actually not a race condition issue (otherwise please would you please explain what race condition you are seeing), because it looks that a single thread would run into the same problem as explained in my comment linked above. So seems to me that the solution would be to fix MetricsSourceAdapter#updateAttrCache to let it check whether lastRecs is NULL, and not to access it if so. Does it make sense to you? Thanks.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Brahma Reddy Battula, Thank you and other folks for working on the issue here. Thanks Tsuyoshi Ozawa for pointing me to this jira since I ran into the same issue and reported HADOOP-13339.

          I have a comment here:

          https://issues.apache.org/jira/browse/HADOOP-13339?focusedCommentId=15361880&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15361880

          Wonder if it makes sense to you guys.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Brahma Reddy Battula , Thank you and other folks for working on the issue here. Thanks Tsuyoshi Ozawa for pointing me to this jira since I ran into the same issue and reported HADOOP-13339 . I have a comment here: https://issues.apache.org/jira/browse/HADOOP-13339?focusedCommentId=15361880&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15361880 Wonder if it makes sense to you guys. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 12s trunk passed
          +1 compile 7m 8s trunk passed
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 22s trunk passed
          +1 javadoc 0m 46s trunk passed
          +1 mvninstall 0m 40s the patch passed
          +1 compile 6m 48s the patch passed
          +1 javac 6m 48s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 33s the patch passed
          +1 javadoc 0m 45s the patch passed
          -1 unit 7m 58s hadoop-common in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          39m 19s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797703/HADOOP-11361-004.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8abd4158cf51 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8b4b525
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 12s trunk passed +1 compile 7m 8s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 22s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 40s the patch passed +1 compile 6m 48s the patch passed +1 javac 6m 48s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 33s the patch passed +1 javadoc 0m 45s the patch passed -1 unit 7m 58s hadoop-common in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 39m 19s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797703/HADOOP-11361-004.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8abd4158cf51 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8b4b525 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9922/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Sorry for the delay. Reviewing.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Sorry for the delay. Reviewing.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -
          Show
          brahmareddy Brahma Reddy Battula added a comment - Pinging Tsuyoshi Ozawa
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          jmxCacheTS can be 0 only at the beginning.
          At that time, lastRecs also will be null only.
          So it wont break anything

          Morever, parameter 'all' is always passed as true.

          Could you explain why you think it's okay?

          At the time when this issue was raised, that time 'lastRecsCleared' was not present.
          Now since that is present to track whether it was cleared or not, making 'lastRecs' null may not be necessary.

          Show
          brahmareddy Brahma Reddy Battula added a comment - jmxCacheTS can be 0 only at the beginning. At that time, lastRecs also will be null only. So it wont break anything Morever, parameter 'all' is always passed as true. Could you explain why you think it's okay? At the time when this issue was raised, that time 'lastRecsCleared' was not present. Now since that is present to track whether it was cleared or not, making 'lastRecs' null may not be necessary.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Brahma Reddy Battula IIUC, the latest patch looks to break following condition. Could you explain why you think it's okay?

              synchronized(this) {
                if (lastRecs == null && jmxCacheTS == 0) {
                  all = true; // Get all the metrics to populate the sink caches
                }
              }
          
          Show
          ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula IIUC, the latest patch looks to break following condition. Could you explain why you think it's okay? synchronized ( this ) { if (lastRecs == null && jmxCacheTS == 0) { all = true ; // Get all the metrics to populate the sink caches } }
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 30s trunk passed
          +1 compile 5m 37s trunk passed with JDK v1.8.0_77
          +1 compile 6m 32s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 33s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 5m 39s the patch passed with JDK v1.8.0_77
          +1 javac 5m 39s the patch passed
          +1 compile 6m 42s the patch passed with JDK v1.7.0_95
          +1 javac 6m 42s the patch passed
          +1 checkstyle 0m 21s the patch passed
          +1 mvnsite 0m 55s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 48s the patch passed
          +1 javadoc 0m 53s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95
          +1 unit 7m 49s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 54s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          59m 11s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797703/HADOOP-11361-004.patch
          JIRA Issue HADOOP-11361
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux bb1994d65b8b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 594c70f
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9044/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9044/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 30s trunk passed +1 compile 5m 37s trunk passed with JDK v1.8.0_77 +1 compile 6m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 33s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 5m 39s the patch passed with JDK v1.8.0_77 +1 javac 5m 39s the patch passed +1 compile 6m 42s the patch passed with JDK v1.7.0_95 +1 javac 6m 42s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 53s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95 +1 unit 7m 49s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 7m 54s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 59m 11s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797703/HADOOP-11361-004.patch JIRA Issue HADOOP-11361 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bb1994d65b8b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 594c70f Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9044/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9044/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Updated the patch.

          Even after HADOOP-12482, same problem exists with multiple threads.
          But, after HADOOP-12482, making lastRecs null is not necessary.
          Which automatically fixes this race condition.

          Please give your thoughts on this.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Updated the patch. Even after HADOOP-12482 , same problem exists with multiple threads. But, after HADOOP-12482 , making lastRecs null is not necessary. Which automatically fixes this race condition. Please give your thoughts on this.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #639 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/639/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #639 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/639/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1449 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1449/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1449 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1449/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #726 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/726/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #726 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/726/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2657 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2657/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2657 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2657/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #715 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/715/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #715 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/715/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8878/)
          Revert "HADOOP-11361. Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8878/ ) Revert " HADOOP-11361 . Fix a race condition in (jlowe: rev 17b1a5482b32dab82225e8233648990bc77674ba) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          jlowe Jason Lowe added a comment -

          Reverted from trunk, branch-2, and branch-2.7. I left it targeted to 2.8 and 2.7.3 assuming the original problem reported by this JIRA is still an issue.

          Show
          jlowe Jason Lowe added a comment - Reverted from trunk, branch-2, and branch-2.7. I left it targeted to 2.8 and 2.7.3 assuming the original problem reported by this JIRA is still an issue.
          Hide
          jlowe Jason Lowe added a comment -

          Based on initial patch in HADOOP-12594 and earlier comments, I'm reverting this.

          Show
          jlowe Jason Lowe added a comment - Based on initial patch in HADOOP-12594 and earlier comments, I'm reverting this.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 on revert

          Show
          kihwal Kihwal Lee added a comment - +1 on revert
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Jason Lowe Thats nice catch!! Agree with you.I feel , it can be reverted.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Jason Lowe Thats nice catch!! Agree with you.I feel , it can be reverted.
          Hide
          jlowe Jason Lowe added a comment -

          It appears this change can lead to a deadlock in the metrics subsystem. See HADOOP-12594. This change allows the MetricsSourceAdapter to hold a lock on itself while trying to get a lock on the MetricsSystemImpl.

          Note that prior to this change the getMetrics method was careful to only grab the lock outside of the source.getMetrics call, but now the method is being invoked while holding the adapter lock. That ends up grabbing the locks in the opposite order that the metrics system does and we can deadlock.

          Show
          jlowe Jason Lowe added a comment - It appears this change can lead to a deadlock in the metrics subsystem. See HADOOP-12594 . This change allows the MetricsSourceAdapter to hold a lock on itself while trying to get a lock on the MetricsSystemImpl. Note that prior to this change the getMetrics method was careful to only grab the lock outside of the source.getMetrics call, but now the method is being invoked while holding the adapter lock. That ends up grabbing the locks in the opposite order that the metrics system does and we can deadlock.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2543 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2543/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2543 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2543/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #606 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/606/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #606 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/606/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #667 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/667/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #667 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/667/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1403 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1403/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1403 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1403/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #679 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/679/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #679 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/679/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2608 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2608/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2608 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2608/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8804 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8804/)
          Move HADOOP-11361, HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8804 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8804/ ) Move HADOOP-11361 , HADOOP-12348 and HADOOP-12482 from 2.8.0 to 2.7.3 in (ozawa: rev c753617a48bffed491b9ca7a5ca6b3d2df5721bf) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Cherrypicked this to branch-2.7 for HADOOP-12482.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Cherrypicked this to branch-2.7 for HADOOP-12482 .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2143/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2143/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/185/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/185/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2125 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2125/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2125 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2125/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/195/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/195/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #927 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/927/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #927 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/927/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot Tsuyoshi Ozawa for committing and reviewing.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot Tsuyoshi Ozawa for committing and reviewing.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7826 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7826/)
          HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7826 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7826/ ) HADOOP-11361 . Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Brahma Reddy Battula. (ozawa: rev 4356e8a5ef0ac6d11a34704b80ef360a710e623a) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk and branch-2. Thanks Brahma Reddy Battula for your reporting and contribution!

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Brahma Reddy Battula for your reporting and contribution!
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          +1, committing this shortly.

          Show
          ozawa Tsuyoshi Ozawa added a comment - +1, committing this shortly.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 59s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 36s There were no new javac warning messages.
          +1 javadoc 9m 49s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 5s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 5s Tests passed in hadoop-common.
              60m 51s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12732570/HADOOP-111361-003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 065d8f2
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 59s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 5s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 42s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 5s Tests passed in hadoop-common.     60m 51s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732570/HADOOP-111361-003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 065d8f2 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6686/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Tsuyoshi OzawaUpdated the patch, kindly Review,, thanks

          Show
          brahmareddy Brahma Reddy Battula added a comment - Tsuyoshi Ozawa Updated the patch, kindly Review,, thanks
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Brahma Reddy Battula thank you for updating. Fix itself looks good to me. However, the indentation looks wrong. Please correct it. +1 if it's addressed.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula thank you for updating. Fix itself looks good to me. However, the indentation looks wrong. Please correct it. +1 if it's addressed.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 38s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 8s The applied patch generated 6 new checkstyle issues (total was 2426, now 2429).
          -1 whitespace 0m 6s The patch has 39 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 3s Tests passed in hadoop-common.
              60m 24s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731061/HADOOP-11361-002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4c7b9b6
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 8s The applied patch generated 6 new checkstyle issues (total was 2426, now 2429). -1 whitespace 0m 6s The patch has 39 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 3s Tests passed in hadoop-common.     60m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731061/HADOOP-11361-002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c7b9b6 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6512/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks for taking look into this issue..Make sense to me..will move synchronized block and make one synchronized block in updateJmxCache() method...

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks for taking look into this issue..Make sense to me..will move synchronized block and make one synchronized block in updateJmxCache() method...
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Brahma Reddy Battula This problem indicates that MetricsSourceAdapter has a critical section about lastRecs. I think we should move a following if statement into synchronized block. What do you think?

            private void updateJmxCache() {
              ...
              if (getAllMetrics) {
                MetricsCollectorImpl builder = new MetricsCollectorImpl();
                getMetrics(builder, true);
              } // This block should be moved into following synchronized block.
          
              synchronized(this) {
                updateAttrCache();
                if (getAllMetrics) {
                  updateInfoCache();
                }
                jmxCacheTS = Time.now();
                lastRecs = null;  // in case regular interval update is not running
              }
            }
          
          Show
          ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula This problem indicates that MetricsSourceAdapter has a critical section about lastRecs. I think we should move a following if statement into synchronized block. What do you think? private void updateJmxCache() { ... if (getAllMetrics) { MetricsCollectorImpl builder = new MetricsCollectorImpl(); getMetrics(builder, true ); } // This block should be moved into following synchronized block. synchronized ( this ) { updateAttrCache(); if (getAllMetrics) { updateInfoCache(); } jmxCacheTS = Time.now(); lastRecs = null ; // in case regular interval update is not running } }
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 3s The applied patch generated 4 new checkstyle issues (total was 4, now 8).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 31s Tests passed in hadoop-common.
              60m 24s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / a319771
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 3s The applied patch generated 4 new checkstyle issues (total was 4, now 8). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 31s Tests passed in hadoop-common.     60m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a319771 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6462/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 44s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 32s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 3s The applied patch generated 4 new checkstyle issues (total was 4, now 8).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 22m 33s Tests passed in hadoop-common.
              59m 43s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e8d0ee5
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/artifact/patchprocess/diffcheckstylehadoop-common.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 3s The applied patch generated 4 new checkstyle issues (total was 4, now 8). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 33s Tests passed in hadoop-common.     59m 43s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e8d0ee5 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6452/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch
          against trunk revision 144da2e.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 65 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch against trunk revision 144da2e. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 65 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5191//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch
          against trunk revision 8963515.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 65 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685758/HADOOP-11361.patch against trunk revision 8963515. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 65 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5190//console This message is automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          I moved to common..

          Show
          brahmareddy Brahma Reddy Battula added a comment - I moved to common..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          does somebody move to HADOOP COMMON..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - does somebody move to HADOOP COMMON..?

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development