Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11180

Intermittent deadlock in NameNode when failover happens.

    Details

      Description

      It is happening due to metrics getting updated at the same time when failover is happening. Please find attached jstack at that point of time.

      1. HDFS-11180.00.patch
        3 kB
        Akira Ajisaka
      2. HDFS-11180.01.patch
        3 kB
        Akira Ajisaka
      3. HDFS-11180.02.patch
        4 kB
        Akira Ajisaka
      4. HDFS-11180.03.patch
        9 kB
        Akira Ajisaka
      5. HDFS-11180.04.patch
        16 kB
        Akira Ajisaka
      6. HDFS-11180-branch-2.01.patch
        15 kB
        Akira Ajisaka
      7. HDFS-11180-branch-2.6.01.patch
        13 kB
        Akira Ajisaka
      8. HDFS-11180-branch-2.7.01.patch
        13 kB
        Akira Ajisaka
      9. HDFS-11180-branch-2.8.01.patch
        15 kB
        Akira Ajisaka
      10. jstack.log
        107 kB
        Abhishek Modi

        Issue Links

          Activity

          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Abhishek Modi, thank you for reporting this. Would you tell us what version of Apache Hadoop?

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Abhishek Modi , thank you for reporting this. Would you tell us what version of Apache Hadoop?
          Hide
          abmodi Abhishek Modi added a comment -

          We are using Hadoop-2.6.0

          Show
          abmodi Abhishek Modi added a comment - We are using Hadoop-2.6.0
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for your information.
          It looks like:

          • NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics.
          • At the same time, metrics system holds a lock of MetricsSystemImpl and requires a lock of FSEditLog when publishing FSNameSystem.TransactionsSinceLastCheckpoint metric.

          I'm thinking we don't need to hold a lock when publishing FSNameSystem.TransactionsSinceLastCheckpoint metric.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for your information. It looks like: NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics. At the same time, metrics system holds a lock of MetricsSystemImpl and requires a lock of FSEditLog when publishing FSNameSystem.TransactionsSinceLastCheckpoint metric. I'm thinking we don't need to hold a lock when publishing FSNameSystem.TransactionsSinceLastCheckpoint metric.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Attaching a sample patch.
          I'm thinking removing synchronized from FSNameSystem.getLastWrittenTxId and FSNameSystem.getCurSegmentTxId seems problematic, so I created other methods without locking and use them for metrics.

          Show
          ajisakaa Akira Ajisaka added a comment - Attaching a sample patch. I'm thinking removing synchronized from FSNameSystem.getLastWrittenTxId and FSNameSystem.getCurSegmentTxId seems problematic, so I created other methods without locking and use them for metrics.
          Hide
          abmodi Abhishek Modi added a comment -

          Can we have a separate readWrite lock for txid? We will acquire readLock in FSEditLog.getLastWrittenTxId and writeLock in FSEditLog.beginTransaction and FSEditLog.setNextTxId. This way, we would be safely able to read TxId in FsNamesystem.getTransactionsSinceLastCheckpoint. Thoughts?

          Show
          abmodi Abhishek Modi added a comment - Can we have a separate readWrite lock for txid? We will acquire readLock in FSEditLog.getLastWrittenTxId and writeLock in FSEditLog.beginTransaction and FSEditLog.setNextTxId. This way, we would be safely able to read TxId in FsNamesystem.getTransactionsSinceLastCheckpoint. Thoughts?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Agreed. Using separate readwrite lock makes the metrics correct.

          Show
          ajisakaa Akira Ajisaka added a comment - Agreed. Using separate readwrite lock makes the metrics correct.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          01: Use ReentrantReadWriteLock.

          Show
          ajisakaa Akira Ajisaka added a comment - 01: Use ReentrantReadWriteLock.
          Hide
          ajisakaa Akira Ajisaka added a comment - - edited

          Rethinking this, 01 patch is just removing synchronization.
          Calling synchronized getLastWrittenTxId/getCurSegmentTxId method ensures all other synchronized methods are not being called. Maybe this behavior is avoiding some problems. Therefore now I'm thinking it's not better to remove the synchronization before we conclude that it is safe.

          Show
          ajisakaa Akira Ajisaka added a comment - - edited Rethinking this, 01 patch is just removing synchronization. Calling synchronized getLastWrittenTxId/getCurSegmentTxId method ensures all other synchronized methods are not being called. Maybe this behavior is avoiding some problems. Therefore now I'm thinking it's not better to remove the synchronization before we conclude that it is safe.
          Hide
          abmodi Abhishek Modi added a comment -

          Thanks Akira for quick response. I am not much aware about this code area, so not very sure whether this can cause problem. I will try out this patch in my setup and check whether it causes any issues.

          Show
          abmodi Abhishek Modi added a comment - Thanks Akira for quick response. I am not much aware about this code area, so not very sure whether this can cause problem. I will try out this patch in my setup and check whether it causes any issues.
          Hide
          kihwal Kihwal Lee added a comment -

          NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics.

          It looks like this deadlock can happen only when the QJM is used.

          Metrics update does not need a precise txid. We could introduce unsynchronized methods for metrics and perhaps use volatile for txid?

          Show
          kihwal Kihwal Lee added a comment - NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics. It looks like this deadlock can happen only when the QJM is used. Metrics update does not need a precise txid. We could introduce unsynchronized methods for metrics and perhaps use volatile for txid?
          Hide
          ajisakaa Akira Ajisaka added a comment - - edited

          Thanks Abhishek and Kihwal for the comments. Agreed to make them volatile.
          Attaching 02 patch: 00 patch + make the txids volatile

          Show
          ajisakaa Akira Ajisaka added a comment - - edited Thanks Abhishek and Kihwal for the comments. Agreed to make them volatile. Attaching 02 patch: 00 patch + make the txids volatile
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Cancelling patch because we need to create unsynchronized getTotalSyncCount() as well.

          Show
          ajisakaa Akira Ajisaka added a comment - Cancelling patch because we need to create unsynchronized getTotalSyncCount() as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 26s trunk passed
          +1 compile 0m 47s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 1m 4s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 55s trunk passed
          +1 javadoc 0m 44s trunk passed
          +1 mvninstall 1m 1s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 1m 3s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 56s hadoop-hdfs-project/hadoop-hdfs generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          +1 javadoc 0m 40s the patch passed
          -1 unit 93m 42s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          114m 51s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs
            Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.state; locked 96% of time Unsynchronized access at FSEditLog.java:96% of time Unsynchronized access at FSEditLog.java:[line 550]
            Increment of volatile field org.apache.hadoop.hdfs.server.namenode.FSEditLog.txid in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:[line 498]
          Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840665/HDFS-11180.02.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 05e9d2dfbf85 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a2b1ff0
          Default Java 1.8.0_111
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17678/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17678/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17678/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17678/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 26s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 0m 44s trunk passed +1 mvninstall 1m 1s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 56s hadoop-hdfs-project/hadoop-hdfs generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) +1 javadoc 0m 40s the patch passed -1 unit 93m 42s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 114m 51s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.state; locked 96% of time Unsynchronized access at FSEditLog.java:96% of time Unsynchronized access at FSEditLog.java: [line 550]   Increment of volatile field org.apache.hadoop.hdfs.server.namenode.FSEditLog.txid in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java: [line 498] Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840665/HDFS-11180.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 05e9d2dfbf85 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a2b1ff0 Default Java 1.8.0_111 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17678/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/17678/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17678/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17678/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          03 patch:

          • Added a regression test
          • Updated FSEditLog and some metric related code to pass the regression test
          • TODO: Ignore findbugs warnings
          Show
          ajisakaa Akira Ajisaka added a comment - 03 patch: Added a regression test Updated FSEditLog and some metric related code to pass the regression test TODO: Ignore findbugs warnings
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 10m 54s trunk passed
          +1 compile 1m 3s trunk passed
          +1 checkstyle 0m 44s trunk passed
          +1 mvnsite 1m 7s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 0m 43s trunk passed
          +1 mvninstall 0m 55s the patch passed
          +1 compile 0m 49s the patch passed
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 31s the patch passed
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 59s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)
          +1 javadoc 0m 40s the patch passed
          -1 unit 108m 51s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          134m 0s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs
            Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.editLogStream; locked 78% of time Unsynchronized access at FSEditLog.java:78% of time Unsynchronized access at FSEditLog.java:[line 1779]
            Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.state; locked 88% of time Unsynchronized access at FSEditLog.java:88% of time Unsynchronized access at FSEditLog.java:[line 571]
            Increment of volatile field org.apache.hadoop.hdfs.server.namenode.FSEditLog.txid in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:[line 519]
          Failed junit tests hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840741/HDFS-11180.03.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b4d9efddde63 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 62b42ef
          Default Java 1.8.0_111
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17683/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17683/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17683/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17683/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 10m 54s trunk passed +1 compile 1m 3s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 55s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 31s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 59s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) +1 javadoc 0m 40s the patch passed -1 unit 108m 51s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 134m 0s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.editLogStream; locked 78% of time Unsynchronized access at FSEditLog.java:78% of time Unsynchronized access at FSEditLog.java: [line 1779]   Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSEditLog.state; locked 88% of time Unsynchronized access at FSEditLog.java:88% of time Unsynchronized access at FSEditLog.java: [line 571]   Increment of volatile field org.apache.hadoop.hdfs.server.namenode.FSEditLog.txid in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java:in org.apache.hadoop.hdfs.server.namenode.FSEditLog.beginTransaction() At FSEditLog.java: [line 519] Failed junit tests hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840741/HDFS-11180.03.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b4d9efddde63 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 62b42ef Default Java 1.8.0_111 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17683/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/17683/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17683/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17683/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          04 patch:

          • Ignored findbugs warnings.
          • There are some metrics using the newly unsynchronized FSEditLog methods and the metrics are used by non-metrics code. I created some new methods using synchronized FSEditLog methods for non-metrics code.
          • Removed try-with-resources statement for regression test. This makes easier for backporting to branch-2.6/2.7.
          • Fixed some issues, such as calling Preconditions.checkState in unsynchronized method.
          Show
          ajisakaa Akira Ajisaka added a comment - 04 patch: Ignored findbugs warnings. There are some metrics using the newly unsynchronized FSEditLog methods and the metrics are used by non-metrics code. I created some new methods using synchronized FSEditLog methods for non-metrics code. Removed try-with-resources statement for regression test. This makes easier for backporting to branch-2.6/2.7. Fixed some issues, such as calling Preconditions.checkState in unsynchronized method.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 30s trunk passed
          +1 compile 0m 57s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 4s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 53s trunk passed
          +1 javadoc 0m 43s trunk passed
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 39s the patch passed
          -1 unit 105m 40s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          127m 9s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840826/HDFS-11180.04.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 61f9089086f8 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0945125
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17698/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17698/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17698/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 30s trunk passed +1 compile 0m 57s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 53s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 105m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 127m 9s Reason Tests Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.hdfs.server.datanode.TestDirectoryScanner Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840826/HDFS-11180.04.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 61f9089086f8 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0945125 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17698/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17698/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17698/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          This patch is now ready for review.

          • TestDirectoryScanner, TestNameNodeMetrics: Probably broken by HDFS-5517. I commented on the issue.
          • TestReadStripedFileWithMissingBlocks, TestBlockStoragePolicy, TestLeaseRecovery2: Passed on my local env.
          Show
          ajisakaa Akira Ajisaka added a comment - This patch is now ready for review. TestDirectoryScanner, TestNameNodeMetrics: Probably broken by HDFS-5517 . I commented on the issue. TestReadStripedFileWithMissingBlocks, TestBlockStoragePolicy, TestLeaseRecovery2: Passed on my local env.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics.

          It looks like this deadlock can happen only when the QJM is used.

          In addition, this deadlock can happen only when there is at least one metrics sink registered to MetricsSystem. If there is no metrics sink, MetricsSystemImpl.sampleMetrics can not be called.

          Show
          ajisakaa Akira Ajisaka added a comment - NameNode holds a lock of FSEditLog and requires a lock of MetricsSystemImpl when registering IPCLoggerChannel metrics. It looks like this deadlock can happen only when the QJM is used. In addition, this deadlock can happen only when there is at least one metrics sink registered to MetricsSystem. If there is no metrics sink, MetricsSystemImpl.sampleMetrics can not be called.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 for the patch.

          I am not sure whether it is worth mentioning the correctness of the value in the release note. For most metrics uses, if not all, the values won't be practically no more "inaccurate" than before.

          Show
          kihwal Kihwal Lee added a comment - +1 for the patch. I am not sure whether it is worth mentioning the correctness of the value in the release note. For most metrics uses, if not all, the values won't be practically no more "inaccurate" than before.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Kihwal Lee for reviewing my patch.

          For most metrics uses, if not all, the values won't be practically no more "inaccurate" than before.

          Agreed. Removing the release note.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Kihwal Lee for reviewing my patch. For most metrics uses, if not all, the values won't be practically no more "inaccurate" than before. Agreed. Removing the release note.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Rebased for branch-2.
          I'll commit 04 patch to trunk tonight JST if there is no objection.

          Show
          ajisakaa Akira Ajisaka added a comment - Rebased for branch-2. I'll commit 04 patch to trunk tonight JST if there is no objection.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 18m 31s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 48s branch-2 passed
          +1 compile 0m 47s branch-2 passed with JDK v1.8.0_111
          +1 compile 0m 43s branch-2 passed with JDK v1.7.0_121
          +1 checkstyle 0m 33s branch-2 passed
          +1 mvnsite 1m 4s branch-2 passed
          +1 mvneclipse 0m 24s branch-2 passed
          +1 findbugs 2m 18s branch-2 passed
          +1 javadoc 1m 6s branch-2 passed with JDK v1.8.0_111
          +1 javadoc 1m 47s branch-2 passed with JDK v1.7.0_121
          +1 mvninstall 1m 0s the patch passed
          +1 compile 0m 53s the patch passed with JDK v1.8.0_111
          +1 javac 0m 53s the patch passed
          +1 compile 0m 51s the patch passed with JDK v1.7.0_121
          +1 javac 0m 51s the patch passed
          +1 checkstyle 0m 36s the patch passed
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 2m 24s the patch passed
          +1 javadoc 1m 0s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 37s the patch passed with JDK v1.7.0_121
          -1 unit 69m 3s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          199m 50s



          Reason Tests
          JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.TestFileAppendRestart
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.TestCacheDirectives



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841219/HDFS-11180-branch-2.01.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux dd0b2ca5263e 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 8ffe86f
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17726/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17726/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17726/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 18m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 48s branch-2 passed +1 compile 0m 47s branch-2 passed with JDK v1.8.0_111 +1 compile 0m 43s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 0m 33s branch-2 passed +1 mvnsite 1m 4s branch-2 passed +1 mvneclipse 0m 24s branch-2 passed +1 findbugs 2m 18s branch-2 passed +1 javadoc 1m 6s branch-2 passed with JDK v1.8.0_111 +1 javadoc 1m 47s branch-2 passed with JDK v1.7.0_121 +1 mvninstall 1m 0s the patch passed +1 compile 0m 53s the patch passed with JDK v1.8.0_111 +1 javac 0m 53s the patch passed +1 compile 0m 51s the patch passed with JDK v1.7.0_121 +1 javac 0m 51s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 24s the patch passed +1 javadoc 1m 0s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_121 -1 unit 69m 3s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 199m 50s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestCacheDirectives Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841219/HDFS-11180-branch-2.01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dd0b2ca5263e 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 8ffe86f Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17726/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17726/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17726/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed v4 patch to trunk.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed v4 patch to trunk.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The precommit job for branch-2 looks good. The failed tests passed locally.

          Show
          ajisakaa Akira Ajisaka added a comment - The precommit job for branch-2 looks good. The failed tests passed locally.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The branch-2 patch can be applied to branch-2.8 cleanly. Renamed the patch to see what happens on branch-2.8.

          Show
          ajisakaa Akira Ajisaka added a comment - The branch-2 patch can be applied to branch-2.8 cleanly. Renamed the patch to see what happens on branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10923/)
          HDFS-11180. Intermittent deadlock in NameNode when failover happens. (aajisaka: rev e0fa49234fd37aca88e1caa95bac77bca192bae4)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyCheckpointer.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemMBean.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10923/ ) HDFS-11180 . Intermittent deadlock in NameNode when failover happens. (aajisaka: rev e0fa49234fd37aca88e1caa95bac77bca192bae4) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java (edit) hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyCheckpointer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemMBean.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 52s branch-2.8 passed
          +1 compile 0m 39s branch-2.8 passed with JDK v1.8.0_111
          +1 compile 0m 43s branch-2.8 passed with JDK v1.7.0_121
          +1 checkstyle 0m 27s branch-2.8 passed
          +1 mvnsite 0m 53s branch-2.8 passed
          +1 mvneclipse 0m 17s branch-2.8 passed
          +1 findbugs 2m 29s branch-2.8 passed
          +1 javadoc 2m 9s branch-2.8 passed with JDK v1.8.0_111
          +1 javadoc 3m 4s branch-2.8 passed with JDK v1.7.0_121
          +1 mvninstall 1m 22s the patch passed
          +1 compile 1m 39s the patch passed with JDK v1.8.0_111
          +1 javac 1m 39s the patch passed
          +1 compile 1m 19s the patch passed with JDK v1.7.0_121
          +1 javac 1m 19s the patch passed
          -0 checkstyle 1m 3s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 476 unchanged - 2 fixed = 477 total (was 478)
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 2m 38s the patch passed
          +1 javadoc 0m 50s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 30s the patch passed with JDK v1.7.0_121
          -1 unit 54m 57s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          153m 42s



          Reason Tests
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.TestCrcCorruption



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841291/HDFS-11180-branch-2.8.01.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux cda78941ad47 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / dd4aceb
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17733/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17733/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17733/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17733/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 52s branch-2.8 passed +1 compile 0m 39s branch-2.8 passed with JDK v1.8.0_111 +1 compile 0m 43s branch-2.8 passed with JDK v1.7.0_121 +1 checkstyle 0m 27s branch-2.8 passed +1 mvnsite 0m 53s branch-2.8 passed +1 mvneclipse 0m 17s branch-2.8 passed +1 findbugs 2m 29s branch-2.8 passed +1 javadoc 2m 9s branch-2.8 passed with JDK v1.8.0_111 +1 javadoc 3m 4s branch-2.8 passed with JDK v1.7.0_121 +1 mvninstall 1m 22s the patch passed +1 compile 1m 39s the patch passed with JDK v1.8.0_111 +1 javac 1m 39s the patch passed +1 compile 1m 19s the patch passed with JDK v1.7.0_121 +1 javac 1m 19s the patch passed -0 checkstyle 1m 3s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 476 unchanged - 2 fixed = 477 total (was 478) +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 38s the patch passed +1 javadoc 0m 50s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 30s the patch passed with JDK v1.7.0_121 -1 unit 54m 57s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 153m 42s Reason Tests JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.TestCrcCorruption Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841291/HDFS-11180-branch-2.8.01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cda78941ad47 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / dd4aceb Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17733/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17733/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17733/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17733/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I ran the failed tests locally and they passed. I'll commit branch-2 patch to branch-2/2.8.

          Show
          ajisakaa Akira Ajisaka added a comment - I ran the failed tests locally and they passed. I'll commit branch-2 patch to branch-2/2.8.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed branch-2 patch to branch-2/2.8. I'll provide a branch-2.7 patch shortly.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed branch-2 patch to branch-2/2.8. I'll provide a branch-2.7 patch shortly.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 39s branch-2.6 passed
          -1 compile 0m 49s hadoop-hdfs in branch-2.6 failed with JDK v1.8.0_111.
          -1 compile 0m 50s hadoop-hdfs in branch-2.6 failed with JDK v1.7.0_121.
          +1 checkstyle 0m 25s branch-2.6 passed
          +1 mvnsite 1m 2s branch-2.6 passed
          +1 mvneclipse 0m 19s branch-2.6 passed
          -1 findbugs 3m 11s hadoop-hdfs-project/hadoop-hdfs in branch-2.6 has 272 extant Findbugs warnings.
          +1 javadoc 1m 20s branch-2.6 passed with JDK v1.8.0_111
          +1 javadoc 1m 55s branch-2.6 passed with JDK v1.7.0_121
          +1 mvninstall 0m 56s the patch passed
          -1 compile 0m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_111.
          -1 javac 0m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_111.
          -1 compile 0m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          -1 javac 0m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 checkstyle 0m 22s the patch passed
          +1 mvnsite 1m 4s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          -1 whitespace 0m 0s The patch has 2996 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 1m 12s The patch 106 line(s) with tabs.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 18s the patch passed
          +1 javadoc 1m 15s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 56s the patch passed with JDK v1.7.0_121
          -1 unit 0m 47s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          -1 asflicense 0m 39s The patch generated 76 ASF License warnings.
          48m 55s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841407/HDFS-11180-branch-2.6.01.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b780ca7bd0f6 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.6 / c2d936f
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          findbugs v1.3.9
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17738/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17738/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 39s branch-2.6 passed -1 compile 0m 49s hadoop-hdfs in branch-2.6 failed with JDK v1.8.0_111. -1 compile 0m 50s hadoop-hdfs in branch-2.6 failed with JDK v1.7.0_121. +1 checkstyle 0m 25s branch-2.6 passed +1 mvnsite 1m 2s branch-2.6 passed +1 mvneclipse 0m 19s branch-2.6 passed -1 findbugs 3m 11s hadoop-hdfs-project/hadoop-hdfs in branch-2.6 has 272 extant Findbugs warnings. +1 javadoc 1m 20s branch-2.6 passed with JDK v1.8.0_111 +1 javadoc 1m 55s branch-2.6 passed with JDK v1.7.0_121 +1 mvninstall 0m 56s the patch passed -1 compile 0m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_111. -1 javac 0m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_111. -1 compile 0m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_121. -1 javac 0m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 2996 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 12s The patch 106 line(s) with tabs. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 18s the patch passed +1 javadoc 1m 15s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 56s the patch passed with JDK v1.7.0_121 -1 unit 0m 47s hadoop-hdfs in the patch failed with JDK v1.7.0_121. -1 asflicense 0m 39s The patch generated 76 ASF License warnings. 48m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841407/HDFS-11180-branch-2.6.01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b780ca7bd0f6 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.6 / c2d936f Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt findbugs v1.3.9 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17738/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17738/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17738/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 35s branch-2.6 passed
          -1 compile 0m 41s hadoop-hdfs in branch-2.6 failed with JDK v1.8.0_111.
          -1 compile 0m 44s hadoop-hdfs in branch-2.6 failed with JDK v1.7.0_121.
          +1 checkstyle 0m 22s branch-2.6 passed
          +1 mvnsite 0m 56s branch-2.6 passed
          +1 mvneclipse 0m 17s branch-2.6 passed
          -1 findbugs 2m 50s hadoop-hdfs-project/hadoop-hdfs in branch-2.6 has 273 extant Findbugs warnings.
          +1 javadoc 1m 9s branch-2.6 passed with JDK v1.8.0_111
          +1 javadoc 1m 49s branch-2.6 passed with JDK v1.7.0_121
          +1 mvninstall 0m 52s the patch passed
          -1 compile 0m 40s hadoop-hdfs in the patch failed with JDK v1.8.0_111.
          -1 javac 0m 40s hadoop-hdfs in the patch failed with JDK v1.8.0_111.
          -1 compile 0m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          -1 javac 0m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 checkstyle 0m 19s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          -1 whitespace 0m 0s The patch has 2997 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 1m 16s The patch 106 line(s) with tabs.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 3m 2s the patch passed
          +1 javadoc 1m 4s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 48s the patch passed with JDK v1.7.0_121
          -1 unit 0m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          -1 asflicense 0m 35s The patch generated 76 ASF License warnings.
          43m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:44eef0e
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841407/HDFS-11180-branch-2.6.01.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1c4281128183 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.6 / c2d936f
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          findbugs v1.3.9
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17739/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17739/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 35s branch-2.6 passed -1 compile 0m 41s hadoop-hdfs in branch-2.6 failed with JDK v1.8.0_111. -1 compile 0m 44s hadoop-hdfs in branch-2.6 failed with JDK v1.7.0_121. +1 checkstyle 0m 22s branch-2.6 passed +1 mvnsite 0m 56s branch-2.6 passed +1 mvneclipse 0m 17s branch-2.6 passed -1 findbugs 2m 50s hadoop-hdfs-project/hadoop-hdfs in branch-2.6 has 273 extant Findbugs warnings. +1 javadoc 1m 9s branch-2.6 passed with JDK v1.8.0_111 +1 javadoc 1m 49s branch-2.6 passed with JDK v1.7.0_121 +1 mvninstall 0m 52s the patch passed -1 compile 0m 40s hadoop-hdfs in the patch failed with JDK v1.8.0_111. -1 javac 0m 40s hadoop-hdfs in the patch failed with JDK v1.8.0_111. -1 compile 0m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_121. -1 javac 0m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 checkstyle 0m 19s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 2997 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 16s The patch 106 line(s) with tabs. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 2s the patch passed +1 javadoc 1m 4s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 48s the patch passed with JDK v1.7.0_121 -1 unit 0m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_121. -1 asflicense 0m 35s The patch generated 76 ASF License warnings. 43m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:44eef0e JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841407/HDFS-11180-branch-2.6.01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1c4281128183 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.6 / c2d936f Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt findbugs v1.3.9 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_111.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17739/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17739/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17739/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 43s branch-2.7 passed
          +1 compile 1m 1s branch-2.7 passed with JDK v1.8.0_111
          +1 compile 0m 59s branch-2.7 passed with JDK v1.7.0_121
          +1 checkstyle 0m 32s branch-2.7 passed
          +1 mvnsite 0m 59s branch-2.7 passed
          +1 mvneclipse 0m 16s branch-2.7 passed
          +1 findbugs 2m 54s branch-2.7 passed
          +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_111
          +1 javadoc 1m 44s branch-2.7 passed with JDK v1.7.0_121
          +1 mvninstall 0m 51s the patch passed
          +1 compile 0m 54s the patch passed with JDK v1.8.0_111
          +1 javac 0m 54s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.7.0_121
          +1 javac 0m 59s the patch passed
          -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 565 unchanged - 2 fixed = 566 total (was 567)
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          -1 whitespace 0m 0s The patch has 3590 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 1m 29s The patch 106 line(s) with tabs.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 4s the patch passed
          +1 javadoc 0m 55s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 42s the patch passed with JDK v1.7.0_121
          -1 unit 43m 19s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          -1 asflicense 0m 20s The patch generated 3 ASF License warnings.
          124m 35s



          Reason Tests
          JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue HDFS-11180
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841413/HDFS-11180-branch-2.7.01.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d955c97d4cf4 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 45bc79d
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17740/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17740/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 43s branch-2.7 passed +1 compile 1m 1s branch-2.7 passed with JDK v1.8.0_111 +1 compile 0m 59s branch-2.7 passed with JDK v1.7.0_121 +1 checkstyle 0m 32s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 16s branch-2.7 passed +1 findbugs 2m 54s branch-2.7 passed +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_111 +1 javadoc 1m 44s branch-2.7 passed with JDK v1.7.0_121 +1 mvninstall 0m 51s the patch passed +1 compile 0m 54s the patch passed with JDK v1.8.0_111 +1 javac 0m 54s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_121 +1 javac 0m 59s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 565 unchanged - 2 fixed = 566 total (was 567) +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 3590 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 29s The patch 106 line(s) with tabs. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 4s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_121 -1 unit 43m 19s hadoop-hdfs in the patch failed with JDK v1.7.0_121. -1 asflicense 0m 20s The patch generated 3 ASF License warnings. 124m 35s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_121 Failed junit tests hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-11180 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841413/HDFS-11180-branch-2.7.01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d955c97d4cf4 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 45bc79d Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17740/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/17740/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17740/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          TestHttpsFileSystem and TestRenameWithSnapshots failed on both JDK7 and JDK8, but didn't fail on my local.

          Show
          ajisakaa Akira Ajisaka added a comment - TestHttpsFileSystem and TestRenameWithSnapshots failed on both JDK7 and JDK8, but didn't fail on my local.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I'll commit the branch-2.7 patch tomorrow if there are no objections.

          Show
          ajisakaa Akira Ajisaka added a comment - I'll commit the branch-2.7 patch tomorrow if there are no objections.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed branch-2.7 patch. I'll run full HDFS tests locally with the branch-2.6 patch and then commit it.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed branch-2.7 patch. I'll run full HDFS tests locally with the branch-2.6 patch and then commit it.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Ran all the HDFS tests and the following tests failed.

          • TestJMXGet: HDFS-9072 (fixed in 2.7.3+ but not fixed in branch-2.6)
          • TestLazyPersistFiles: HDFS-9743 (fixed in 2.7.3+ but not fixed in branch-2.6)

          All other tests passed, so I'll commit this.

          Show
          ajisakaa Akira Ajisaka added a comment - Ran all the HDFS tests and the following tests failed. TestJMXGet: HDFS-9072 (fixed in 2.7.3+ but not fixed in branch-2.6) TestLazyPersistFiles: HDFS-9743 (fixed in 2.7.3+ but not fixed in branch-2.6) All other tests passed, so I'll commit this.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to branch-2.6. Closing this.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to branch-2.6. Closing this.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          We deeply investigated this issue and found probably this deadlock happens only when dfs.namenode.edits.dir includes JournalNodes which are not included in dfs.namenode.shared.edits.dir. Hi Abhishek Modi, what's your setting?
          Many thanks to Kousuke Saruta and Masatake Iwasaki for the research.

          Show
          ajisakaa Akira Ajisaka added a comment - We deeply investigated this issue and found probably this deadlock happens only when dfs.namenode.edits.dir includes JournalNodes which are not included in dfs.namenode.shared.edits.dir. Hi Abhishek Modi , what's your setting? Many thanks to Kousuke Saruta and Masatake Iwasaki for the research.
          Hide
          abmodi Abhishek Modi added a comment -

          We have set:
          <property>
          <name>dfs.namenode.name.dir</name>
          <value>
          /media/ephemeral0/dfs/namenode,/media/ephemeral1/dfs/namenode
          </value>
          <property>
          <name>dfs.namenode.shared.edits.dir</name>
          <value>
          qjournal://ec2-54-92-238-125.compute-1.amazonaws.com:8485;ec2-54-163-80-110.compute-1.amazonaws.com:8485;ec2-54-197-119-189.compute-1.amazonaws.com:8485/HACluster
          </value>
          </property>

          We just followed the documentation for HA. What is the recommended settings for these parameters?

          Show
          abmodi Abhishek Modi added a comment - We have set: <property> <name>dfs.namenode.name.dir</name> <value> /media/ephemeral0/dfs/namenode,/media/ephemeral1/dfs/namenode </value> <property> <name>dfs.namenode.shared.edits.dir</name> <value> qjournal://ec2-54-92-238-125.compute-1.amazonaws.com:8485;ec2-54-163-80-110.compute-1.amazonaws.com:8485;ec2-54-197-119-189.compute-1.amazonaws.com:8485/HACluster </value> </property> We just followed the documentation for HA. What is the recommended settings for these parameters?
          Hide
          xkrogen Erik Krogen added a comment -

          Akira Ajisaka, shouldn't FSNamesystem#getTransactionsSinceLastCheckpoint also use getLastWrittenTxIdWithoutLock instead of getLastWrittenTxId?

          Show
          xkrogen Erik Krogen added a comment - Akira Ajisaka , shouldn't FSNamesystem#getTransactionsSinceLastCheckpoint also use getLastWrittenTxIdWithoutLock instead of getLastWrittenTxId ?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The following code was not changed in this commit and getLastWrittenTxId is not used. Am I missing something?

          FSNameSystem.java
            @Metric({"TransactionsSinceLastCheckpoint",
                "Number of transactions since last checkpoint"})
            public long getTransactionsSinceLastCheckpoint() {
              return getFSImage().getLastAppliedOrWrittenTxId() -
                  getFSImage().getStorage().getMostRecentCheckpointTxId();
            }
          
          Show
          ajisakaa Akira Ajisaka added a comment - The following code was not changed in this commit and getLastWrittenTxId is not used. Am I missing something? FSNameSystem.java @Metric({ "TransactionsSinceLastCheckpoint" , " Number of transactions since last checkpoint" }) public long getTransactionsSinceLastCheckpoint() { return getFSImage().getLastAppliedOrWrittenTxId() - getFSImage().getStorage().getMostRecentCheckpointTxId(); }
          Hide
          xkrogen Erik Krogen added a comment -

          Akira Ajisaka, sorry, I should have specified - I was looking at branch-2.7. However, even in branch-2 and trunk, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxId, presenting the same problem that to complete the metrics call eventually requires synchronization on the FSEditLog.

          Show
          xkrogen Erik Krogen added a comment - Akira Ajisaka , sorry, I should have specified - I was looking at branch-2.7. However, even in branch-2 and trunk, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxId , presenting the same problem that to complete the metrics call eventually requires synchronization on the FSEditLog .
          Hide
          ajisakaa Akira Ajisaka added a comment -

          However, even in branch-2 and trunk, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxId

          After this commit, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxIdWithoutLock.

          HDFS-11180.04.patch
          @@ -1418,6 +1418,15 @@ public synchronized long getLastAppliedTxId() {
           
             public long getLastAppliedOrWrittenTxId() {
               return Math.max(lastAppliedTxId,
          +        editLog != null ? editLog.getLastWrittenTxIdWithoutLock() : 0);
          +  }
          +
          +  /**
          +   * This method holds a lock of FSEditLog to get the correct value.
          +   * This method must not be used for metrics.
          +   */
          +  public long getCorrectLastAppliedOrWrittenTxId() {
          +    return Math.max(lastAppliedTxId,
                   editLog != null ? editLog.getLastWrittenTxId() : 0);
             }
          

          In addition, I've added a regression test to verify that FSNameSystem metrics don't synchronize FSEditLog. Please see the test for the detail.

          Show
          ajisakaa Akira Ajisaka added a comment - However, even in branch-2 and trunk, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxId After this commit, FSImage#getLastAppliedOrWrittenTxId internally calls FSEditLog#getLastWrittenTxIdWithoutLock. HDFS-11180 .04.patch @@ -1418,6 +1418,15 @@ public synchronized long getLastAppliedTxId() { public long getLastAppliedOrWrittenTxId() { return Math .max(lastAppliedTxId, + editLog != null ? editLog.getLastWrittenTxIdWithoutLock() : 0); + } + + /** + * This method holds a lock of FSEditLog to get the correct value. + * This method must not be used for metrics. + */ + public long getCorrectLastAppliedOrWrittenTxId() { + return Math .max(lastAppliedTxId, editLog != null ? editLog.getLastWrittenTxId() : 0); } In addition, I've added a regression test to verify that FSNameSystem metrics don't synchronize FSEditLog. Please see the test for the detail.
          Hide
          xkrogen Erik Krogen added a comment - - edited

          Ah, my mistake on the branch-2 / trunk. Accidentally had the wrong branch checked out when I was looking in FSImage. Apologies for that red herring.

          My question stands on branch-2.7 - I double-checked I wasn't making the same mistake. I also found the reason that the test is currently succeeding. It appears that the JMX cache is being populated before the lock is taken on the FSEditLog, then once the lock is taken the metrics are able to be read because they are cached (and so the original method requiring synchronization is not used). I confirmed this using the logs available and also if you add a Thread.sleep(10000) (equivalent to the default JMX cache TTL) at the start of the synchronization block in branch-2.7 the test will fail.

          Show
          xkrogen Erik Krogen added a comment - - edited Ah, my mistake on the branch-2 / trunk. Accidentally had the wrong branch checked out when I was looking in FSImage . Apologies for that red herring. My question stands on branch-2.7 - I double-checked I wasn't making the same mistake. I also found the reason that the test is currently succeeding. It appears that the JMX cache is being populated before the lock is taken on the FSEditLog , then once the lock is taken the metrics are able to be read because they are cached (and so the original method requiring synchronization is not used). I confirmed this using the logs available and also if you add a Thread.sleep(10000) (equivalent to the default JMX cache TTL) at the start of the synchronization block in branch-2.7 the test will fail.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I also found the reason that the test is currently succeeding. It appears that the JMX cache is being populated before the lock is taken on the FSEditLog, then once the lock is taken the metrics are able to be read because they are cached (and so the original method requiring synchronization is not used). I confirmed this using the logs available and also if you add a Thread.sleep(10000) (equivalent to the default JMX cache TTL) at the start of the synchronization block in branch-2.7 the test will fail.

          I couldn't reproduce this, but agreed with you. Let's update the tests in a separate jira.

          Show
          ajisakaa Akira Ajisaka added a comment - I also found the reason that the test is currently succeeding. It appears that the JMX cache is being populated before the lock is taken on the FSEditLog, then once the lock is taken the metrics are able to be read because they are cached (and so the original method requiring synchronization is not used). I confirmed this using the logs available and also if you add a Thread.sleep(10000) (equivalent to the default JMX cache TTL) at the start of the synchronization block in branch-2.7 the test will fail. I couldn't reproduce this, but agreed with you. Let's update the tests in a separate jira.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Filed HDFS-11290 for tracking this issue.

          Show
          ajisakaa Akira Ajisaka added a comment - Filed HDFS-11290 for tracking this issue.
          Hide
          xkrogen Erik Krogen added a comment -

          Thanks Akira Ajisaka! I will do a little investigation and see if I can take on that ticket.

          Show
          xkrogen Erik Krogen added a comment - Thanks Akira Ajisaka ! I will do a little investigation and see if I can take on that ticket.

            People

            • Assignee:
              ajisakaa Akira Ajisaka
              Reporter:
              abmodi Abhishek Modi
            • Votes:
              2 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development