Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10609

Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: encryption
    • Labels:
      None
    • Environment:

      CDH5.8.0

    • Hadoop Flags:
      Reviewed
    • Release Note:
      If pipeline recovery fails due to expired encryption key, attempt to refresh the key and retry.

      Description

      In normal operations, if SASL negotiation fails due to InvalidEncryptionKeyException, it is typically a benign exception, which is caught and retried :

      SaslDataTransferServer#doSaslHandshake
        if (ioe instanceof SaslException &&
            ioe.getCause() != null &&
            ioe.getCause() instanceof InvalidEncryptionKeyException) {
          // This could just be because the client is long-lived and hasn't gotten
          // a new encryption key from the NN in a while. Upon receiving this
          // error, the client will get a new encryption key from the NN and retry
          // connecting to this DN.
          sendInvalidKeySaslErrorMessage(out, ioe.getCause().getMessage());
        } 
      
      DFSOutputStream.DataStreamer#createBlockOutputStream
      if (ie instanceof InvalidEncryptionKeyException && refetchEncryptionKey > 0) {
                  DFSClient.LOG.info("Will fetch a new encryption key and retry, " 
                      + "encryption key was invalid when connecting to "
                      + nodes[0] + " : " + ie);
      

      However, if the exception is thrown during pipeline recovery, the corresponding code does not handle it properly, and the exception is spilled out to downstream applications, such as SOLR, aborting its operation:

      2016-07-06 12:12:51,992 ERROR org.apache.solr.update.HdfsTransactionLog: Exception closing tlog.
      org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=557709482) doesn't exist. Current key: 1350592619
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiatedCipherOption(DataTransferSaslUtil.java:417)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:474)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getEncryptedStreams(SaslDataTransferClient.java:299)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:242)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:211)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.socketSend(SaslDataTransferClient.java:183)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.transfer(DFSOutputStream.java:1308)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1272)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1433)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1147)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:632)
      2016-07-06 12:12:51,997 ERROR org.apache.solr.update.CommitTracker: auto commit error...:org.apache.solr.common.SolrException: org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=557709482) doesn't exist. Current key: 1350592619
      at org.apache.solr.update.HdfsTransactionLog.close(HdfsTransactionLog.java:316)
      at org.apache.solr.update.TransactionLog.decref(TransactionLog.java:505)
      at org.apache.solr.update.UpdateLog.addOldLog(UpdateLog.java:380)
      at org.apache.solr.update.UpdateLog.postCommit(UpdateLog.java:676)
      at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:623)
      at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=557709482) doesn't exist. Current key: 1350592619
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiatedCipherOption(DataTransferSaslUtil.java:417)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:474)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getEncryptedStreams(SaslDataTransferClient.java:299)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:242)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:211)
      at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.socketSend(SaslDataTransferClient.java:183)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.transfer(DFSOutputStream.java:1308)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1272)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1433)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1147)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:632)

      This exception should be contained within HDFS, caught and retried just like in createBlockOutputStream()

      1. HDFS-10609.001.patch
        5 kB
        Wei-Chiu Chuang
      2. HDFS-10609.002.patch
        8 kB
        Wei-Chiu Chuang
      3. HDFS-10609.003.patch
        44 kB
        Wei-Chiu Chuang
      4. HDFS-10609.004.patch
        41 kB
        Wei-Chiu Chuang
      5. HDFS-10609.005.patch
        41 kB
        Wei-Chiu Chuang
      6. HDFS-10609.branch-2.7.patch
        40 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        SaslDataTransferClient.socketSend is also used in a number of places in Hadoop. It would be great to also fix these places as well.

        Show
        jojochuang Wei-Chiu Chuang added a comment - SaslDataTransferClient.socketSend is also used in a number of places in Hadoop. It would be great to also fix these places as well.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Patch v01. A simple fix and a test case.

        The test case creates a scenario where a file is present on the cluster, and the cluster's BlockTokenSecretManager is manipulated such that the block's token expires after a short duration. After sleeping for 15 seconds, shutdown the datanode, and the client write to the file again to induce pipeline recovery.

        Note that because the client retries block transfer three times, there are two possible exceptions without the fix:

        • InvalidEncryptionKeyException because the token expires
        • IOException "Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try" because the cluster has only 4 datanodes, and after the first attempt fails due to InvalidEncryptionKeyException which excludes one datanode, the subsequent attempt will see this exception.
        Show
        jojochuang Wei-Chiu Chuang added a comment - Patch v01. A simple fix and a test case. The test case creates a scenario where a file is present on the cluster, and the cluster's BlockTokenSecretManager is manipulated such that the block's token expires after a short duration. After sleeping for 15 seconds, shutdown the datanode, and the client write to the file again to induce pipeline recovery. Note that because the client retries block transfer three times, there are two possible exceptions without the fix: InvalidEncryptionKeyException because the token expires IOException "Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try" because the cluster has only 4 datanodes, and after the first attempt fails due to InvalidEncryptionKeyException which excludes one datanode, the subsequent attempt will see this exception.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 6s Maven dependency ordering for branch
        +1 mvninstall 7m 47s trunk passed
        +1 compile 1m 40s trunk passed
        +1 checkstyle 0m 34s trunk passed
        +1 mvnsite 1m 36s trunk passed
        +1 mvneclipse 0m 29s trunk passed
        +1 findbugs 3m 40s trunk passed
        +1 javadoc 1m 22s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 25s the patch passed
        +1 compile 1m 28s the patch passed
        +1 javac 1m 28s the patch passed
        -0 checkstyle 0m 29s hadoop-hdfs-project: The patch generated 1 new + 95 unchanged - 0 fixed = 96 total (was 95)
        +1 mvnsite 1m 31s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 53s the patch passed
        +1 javadoc 1m 20s the patch passed
        +1 unit 1m 0s hadoop-hdfs-client in the patch passed.
        -1 unit 64m 9s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        95m 5s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestEncryptedTransfer
          hadoop.hdfs.server.datanode.TestDataNodeMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820024/HDFS-10609.001.patch
        JIRA Issue HDFS-10609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 82051ce2aa95 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 703fdf8
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16174/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16174/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16174/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16174/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 7m 47s trunk passed +1 compile 1m 40s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 3m 40s trunk passed +1 javadoc 1m 22s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 25s the patch passed +1 compile 1m 28s the patch passed +1 javac 1m 28s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project: The patch generated 1 new + 95 unchanged - 0 fixed = 96 total (was 95) +1 mvnsite 1m 31s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 53s the patch passed +1 javadoc 1m 20s the patch passed +1 unit 1m 0s hadoop-hdfs-client in the patch passed. -1 unit 64m 9s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 95m 5s Reason Tests Failed junit tests hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.datanode.TestDataNodeMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820024/HDFS-10609.001.patch JIRA Issue HDFS-10609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 82051ce2aa95 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 703fdf8 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16174/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16174/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16174/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16174/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Upload patch v2:

        • Fixed the bug which failed other tests in TestEncryptedTransfer (the files should be created with default replication factor=3)
        • Updated the test case to use 6 data nodes. Because DataStreamer#addDatanode2ExistingPipeline retries 3 times upon exception, using 6 data nodes makes sure that without the fix, the test fails with InvalidEncryptionKeyException, rather than "no more good datanodes" exception.
        • Updated the fix such that it is contained inside DataStreamer#transfer
        Show
        jojochuang Wei-Chiu Chuang added a comment - Upload patch v2: Fixed the bug which failed other tests in TestEncryptedTransfer (the files should be created with default replication factor=3) Updated the test case to use 6 data nodes. Because DataStreamer#addDatanode2ExistingPipeline retries 3 times upon exception, using 6 data nodes makes sure that without the fix, the test fails with InvalidEncryptionKeyException , rather than "no more good datanodes" exception. Updated the fix such that it is contained inside DataStreamer#transfer
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 6s Maven dependency ordering for branch
        +1 mvninstall 7m 24s trunk passed
        +1 compile 1m 33s trunk passed
        +1 checkstyle 0m 33s trunk passed
        +1 mvnsite 1m 33s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 26s trunk passed
        +1 javadoc 1m 21s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 25s the patch passed
        +1 compile 1m 32s the patch passed
        +1 javac 1m 32s the patch passed
        -0 checkstyle 0m 31s hadoop-hdfs-project: The patch generated 1 new + 94 unchanged - 1 fixed = 95 total (was 95)
        +1 mvnsite 1m 32s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 37s the patch passed
        +1 javadoc 1m 16s the patch passed
        +1 unit 0m 58s hadoop-hdfs-client in the patch passed.
        -1 unit 66m 49s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        96m 28s



        Reason Tests
        Failed junit tests hadoop.cli.TestHDFSCLI
        Timed out junit tests org.apache.hadoop.hdfs.TestHdfsAdmin



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820512/HDFS-10609.002.patch
        JIRA Issue HDFS-10609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 83e587d1cc0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 54fe17a
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16214/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16214/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16214/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16214/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 7m 24s trunk passed +1 compile 1m 33s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 33s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 26s trunk passed +1 javadoc 1m 21s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 25s the patch passed +1 compile 1m 32s the patch passed +1 javac 1m 32s the patch passed -0 checkstyle 0m 31s hadoop-hdfs-project: The patch generated 1 new + 94 unchanged - 1 fixed = 95 total (was 95) +1 mvnsite 1m 32s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 37s the patch passed +1 javadoc 1m 16s the patch passed +1 unit 0m 58s hadoop-hdfs-client in the patch passed. -1 unit 66m 49s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 96m 28s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI Timed out junit tests org.apache.hadoop.hdfs.TestHdfsAdmin Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820512/HDFS-10609.002.patch JIRA Issue HDFS-10609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 83e587d1cc0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16214/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16214/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16214/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16214/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        The test failure in TestDFSCli is being tracked by HDFS-10667. TestHdfsAdmin passed in my local tree.

        Other than the test failure and the checkstyle issue, would any watcher like to review the v2 patch? Thanks!

        Show
        jojochuang Wei-Chiu Chuang added a comment - The test failure in TestDFSCli is being tracked by HDFS-10667 . TestHdfsAdmin passed in my local tree. Other than the test failure and the checkstyle issue, would any watcher like to review the v2 patch? Thanks!
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Xiao Chen would you mind to take a look at this patch? Thx!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Xiao Chen would you mind to take a look at this patch? Thx!
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for working on this Wei-Chiu. LGTM, but hoping Andrew Wang could take a quick look.

        Nits:

        • Maybe we don't need to change the log message in createBlockOutputStream
        • We should set the test timeout to a longer value, to avoid false negatives on slow jenkins slaves.
        • Checkstyle could be fixed since we touched that code.

        Searching for InvalidEncryptionKeyException, I see a number of places (e.g. DFSInputStream and DFSStripedInputStream), are those validated for similar fixes?

        Show
        xiaochen Xiao Chen added a comment - Thanks for working on this Wei-Chiu. LGTM, but hoping Andrew Wang could take a quick look. Nits: Maybe we don't need to change the log message in createBlockOutputStream We should set the test timeout to a longer value, to avoid false negatives on slow jenkins slaves. Checkstyle could be fixed since we touched that code. Searching for InvalidEncryptionKeyException , I see a number of places (e.g. DFSInputStream and DFSStripedInputStream ), are those validated for similar fixes?
        Hide
        andrew.wang Andrew Wang added a comment -

        Looks good overall, thanks for working on this Wei-Chiu. A few comments:

        • Is it possible to break the try/catch out into a wrapper function to make the logic more clear? Would also let us avoid the reindentation.
        • The while(true) is also scary, can we instead have it error out after a bounded number of tries?
        • Looks like refetchEncryptionKey could be a boolean, since it's 0 or 1
        • Any way to avoid the sleep in the test? I'm hoping it's possible to trigger the error condition deterministically.
        • Does the test verify that a new key was properly fetched? It's better to assert the behavior you're looking for.
        • Good opportunity to refactor the test class to use @Before and @After to do init and teardown, and avoid the try/catch.
        Show
        andrew.wang Andrew Wang added a comment - Looks good overall, thanks for working on this Wei-Chiu. A few comments: Is it possible to break the try/catch out into a wrapper function to make the logic more clear? Would also let us avoid the reindentation. The while(true) is also scary, can we instead have it error out after a bounded number of tries? Looks like refetchEncryptionKey could be a boolean, since it's 0 or 1 Any way to avoid the sleep in the test? I'm hoping it's possible to trigger the error condition deterministically. Does the test verify that a new key was properly fetched? It's better to assert the behavior you're looking for. Good opportunity to refactor the test class to use @Before and @After to do init and teardown, and avoid the try/catch.
        Hide
        jojochuang Wei-Chiu Chuang added a comment - - edited

        Thanks Xiao Chen and Andrew Wang for reviewing the patch! Took me a while to refactor the code.

        Searching for InvalidEncryptionKeyException, I see a number of places (e.g. DFSInputStream and DFSStripedInputStream), are those validated for similar fixes?

        Given that the exception is thrown when a SASL client connects SaslDataTransferClient#socketSend, there are 7 usage of this method that could potentially throw the exception.
        StripedBlockWriter#init, Dispatcher.PendingMove#dispatch, DataXceiver#replaceBlock, DataXceiver#writeBlock, DataNode.DataTransfer#run

        In fact most of these methods share the same boilerplate code and should probably be extracted into a common method. I have not seen the same exception happening in these places though. These are encryption keys between DataNodes. I am not sure if a new encryption key should be generated in these scenario.

        Is it possible to break the try/catch out into a wrapper function to make the logic more clear?

        Good idea. I refactored most of DataStreamer#transfer into a new class StreamerStreams. I hope this is clearer. Since DataStreamer#createBlockOutputStream shares most of the boilerplate code, I can also refactor that method in a follow up jira.

        Any way to avoid the sleep in the test?

        I've been trying to do this in a less invasive approach, but I ended up using GenericTestUtils#waitFor to detect invalid encryption key. It still uses sleep underneath, but this is still better than an arbitrary sleep period.

        Does the test verify that a new key was properly fetched? It's better to assert the behavior you're looking for.

        Added an assert to ensure the cached encryption key is cleared. The existing logic fetches a new encryption key when there's none available.

        Good opportunity to refactor the test class to use @Before and @After to do init and teardown, and avoid the try/catch.

        Done.

        Show
        jojochuang Wei-Chiu Chuang added a comment - - edited Thanks Xiao Chen and Andrew Wang for reviewing the patch! Took me a while to refactor the code. Searching for InvalidEncryptionKeyException, I see a number of places (e.g. DFSInputStream and DFSStripedInputStream), are those validated for similar fixes? Given that the exception is thrown when a SASL client connects SaslDataTransferClient#socketSend , there are 7 usage of this method that could potentially throw the exception. StripedBlockWriter#init , Dispatcher.PendingMove#dispatch , DataXceiver#replaceBlock , DataXceiver#writeBlock , DataNode.DataTransfer#run In fact most of these methods share the same boilerplate code and should probably be extracted into a common method. I have not seen the same exception happening in these places though. These are encryption keys between DataNodes. I am not sure if a new encryption key should be generated in these scenario. Is it possible to break the try/catch out into a wrapper function to make the logic more clear? Good idea. I refactored most of DataStreamer#transfer into a new class StreamerStreams . I hope this is clearer. Since DataStreamer#createBlockOutputStream shares most of the boilerplate code, I can also refactor that method in a follow up jira. Any way to avoid the sleep in the test? I've been trying to do this in a less invasive approach, but I ended up using GenericTestUtils#waitFor to detect invalid encryption key. It still uses sleep underneath, but this is still better than an arbitrary sleep period. Does the test verify that a new key was properly fetched? It's better to assert the behavior you're looking for. Added an assert to ensure the cached encryption key is cleared. The existing logic fetches a new encryption key when there's none available. Good opportunity to refactor the test class to use @Before and @After to do init and teardown, and avoid the try/catch. Done.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Oops. Forgot to upload the patch. Here's the patch v03. Xiao Chen Andrew Wang could you please take a look at it? Thank you!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Oops. Forgot to upload the patch. Here's the patch v03. Xiao Chen Andrew Wang could you please take a look at it? Thank you!
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        To correct myself –
        The reason InvalidEncryptionKeyException is not seen in other places, is because DataNodes always generate a new encryption key before sasl connection.

        DataXceiver#writeBlock
            ....
                  OutputStream unbufMirrorOut = NetUtils.getOutputStream(mirrorSock,
                      writeTimeout);
                  InputStream unbufMirrorIn = NetUtils.getInputStream(mirrorSock);
                  DataEncryptionKeyFactory keyFactory =
                    datanode.getDataEncryptionKeyFactoryForBlock(block);
                  IOStreamPair saslStreams = datanode.saslClient.socketSend(mirrorSock,
                    unbufMirrorOut, unbufMirrorIn, keyFactory, blockToken, targets[0]);
        
        Show
        jojochuang Wei-Chiu Chuang added a comment - To correct myself – The reason InvalidEncryptionKeyException is not seen in other places, is because DataNodes always generate a new encryption key before sasl connection. DataXceiver#writeBlock .... OutputStream unbufMirrorOut = NetUtils.getOutputStream(mirrorSock, writeTimeout); InputStream unbufMirrorIn = NetUtils.getInputStream(mirrorSock); DataEncryptionKeyFactory keyFactory = datanode.getDataEncryptionKeyFactoryForBlock(block); IOStreamPair saslStreams = datanode.saslClient.socketSend(mirrorSock, unbufMirrorOut, unbufMirrorIn, keyFactory, blockToken, targets[0]);
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 24s Maven dependency ordering for branch
        +1 mvninstall 6m 46s trunk passed
        +1 compile 1m 24s trunk passed
        +1 checkstyle 0m 33s trunk passed
        +1 mvnsite 1m 23s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 3m 3s trunk passed
        +1 javadoc 1m 13s trunk passed
        0 mvndep 0m 6s Maven dependency ordering for patch
        +1 mvninstall 1m 15s the patch passed
        +1 compile 1m 20s the patch passed
        +1 javac 1m 20s the patch passed
        -0 checkstyle 0m 31s hadoop-hdfs-project: The patch generated 11 new + 341 unchanged - 13 fixed = 352 total (was 354)
        +1 mvnsite 1m 17s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 11s the patch passed
        +1 javadoc 1m 9s the patch passed
        +1 unit 0m 52s hadoop-hdfs-client in the patch passed.
        -1 unit 57m 13s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        84m 8s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826462/HDFS-10609.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 638d0b49757e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 20ae1fa
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16595/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16595/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16595/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16595/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 1m 24s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 23s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 3m 3s trunk passed +1 javadoc 1m 13s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 15s the patch passed +1 compile 1m 20s the patch passed +1 javac 1m 20s the patch passed -0 checkstyle 0m 31s hadoop-hdfs-project: The patch generated 11 new + 341 unchanged - 13 fixed = 352 total (was 354) +1 mvnsite 1m 17s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 11s the patch passed +1 javadoc 1m 9s the patch passed +1 unit 0m 52s hadoop-hdfs-client in the patch passed. -1 unit 57m 13s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 84m 8s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826462/HDFS-10609.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 638d0b49757e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 20ae1fa Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16595/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16595/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16595/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16595/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Looks pretty good, but a comment about the retry policy class. I'd prefer to unify the exit condition from the while loop. Right now it could end because of shouldRetry being false or retry throwing an exception. I'd prefer that we always exit by throwing an exception, and from the same place.

        What if we restructured the code something like this:

        do {
          try {
            // do work
          catch (Exception e) {
            policy.failed(src, e);
          } finally {
            // close
          }
        while (policy.continueRetryingOrThrow());
        

        continueRetryingOrThrow is a little verbose, but I think it's important to call out that it's not just doing a boolean check and can throw an Exception which is not expected from a method named something like canRetry or shouldRetry IMO.

        I was also hoping for a little more code de-duplication in the unit test class, since a good chunk of the tests have the initialization steps. Even if we can't put it into a @Before method, we could still break it out into a function. Optional, but something to think about.

        Show
        andrew.wang Andrew Wang added a comment - Looks pretty good, but a comment about the retry policy class. I'd prefer to unify the exit condition from the while loop. Right now it could end because of shouldRetry being false or retry throwing an exception. I'd prefer that we always exit by throwing an exception, and from the same place. What if we restructured the code something like this: do { try { // do work catch (Exception e) { policy.failed(src, e); } finally { // close } while (policy.continueRetryingOrThrow()); continueRetryingOrThrow is a little verbose, but I think it's important to call out that it's not just doing a boolean check and can throw an Exception which is not expected from a method named something like canRetry or shouldRetry IMO. I was also hoping for a little more code de-duplication in the unit test class, since a good chunk of the tests have the initialization steps. Even if we can't put it into a @Before method, we could still break it out into a function. Optional, but something to think about.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks Andrew for your review! I'll work on it soon.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Andrew for your review! I'll work on it soon.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        v04: refactored RefetchEncryptionKeyPolicy following Andrew's review. Also de-dup'ed a big chunk of code in TestEncryptedTransfer.

        Show
        jojochuang Wei-Chiu Chuang added a comment - v04: refactored RefetchEncryptionKeyPolicy following Andrew's review. Also de-dup'ed a big chunk of code in TestEncryptedTransfer.
        Hide
        andrew.wang Andrew Wang added a comment -

        LGTM, that's a lot of code dedupe One little nit, rename fail to failed or recordFailure or unsuccessful? Calling it just fail makes me think of JUnit's Assert.fail() which exits immediately.

        Otherwise +1 pending, thanks for working on this Wei-Chiu!

        Show
        andrew.wang Andrew Wang added a comment - LGTM, that's a lot of code dedupe One little nit, rename fail to failed or recordFailure or unsuccessful ? Calling it just fail makes me think of JUnit's Assert.fail() which exits immediately. Otherwise +1 pending, thanks for working on this Wei-Chiu!
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks a lot Andrew for your quick review! Upload 05 patch to fix the nit you found.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks a lot Andrew for your quick review! Upload 05 patch to fix the nit you found.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 7m 6s trunk passed
        +1 compile 1m 26s trunk passed
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 1m 27s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 3m 10s trunk passed
        +1 javadoc 1m 17s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 18s the patch passed
        +1 compile 1m 23s the patch passed
        +1 javac 1m 23s the patch passed
        +1 checkstyle 0m 32s hadoop-hdfs-project: The patch generated 0 new + 330 unchanged - 11 fixed = 330 total (was 341)
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 25s the patch passed
        +1 javadoc 1m 11s the patch passed
        +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
        -1 unit 58m 22s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        86m 31s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830115/HDFS-10609.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2cb58babfa82 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6eb700e
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16847/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16847/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16847/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 6s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 3m 10s trunk passed +1 javadoc 1m 17s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 18s the patch passed +1 compile 1m 23s the patch passed +1 javac 1m 23s the patch passed +1 checkstyle 0m 32s hadoop-hdfs-project: The patch generated 0 new + 330 unchanged - 11 fixed = 330 total (was 341) +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 25s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 58m 22s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 86m 31s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830115/HDFS-10609.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2cb58babfa82 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16847/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16847/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16847/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 7s Maven dependency ordering for branch
        +1 mvninstall 6m 43s trunk passed
        +1 compile 1m 25s trunk passed
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 1m 26s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 3m 12s trunk passed
        +1 javadoc 1m 15s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 17s the patch passed
        +1 compile 1m 24s the patch passed
        +1 javac 1m 24s the patch passed
        +1 checkstyle 0m 33s hadoop-hdfs-project: The patch generated 0 new + 330 unchanged - 11 fixed = 330 total (was 341)
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 23s the patch passed
        +1 javadoc 1m 12s the patch passed
        +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
        +1 unit 58m 53s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        86m 26s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830115/HDFS-10609.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e168785b063d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6eb700e
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16849/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16849/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 6m 43s trunk passed +1 compile 1m 25s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 3m 12s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 1m 24s the patch passed +1 javac 1m 24s the patch passed +1 checkstyle 0m 33s hadoop-hdfs-project: The patch generated 0 new + 330 unchanged - 11 fixed = 330 total (was 341) +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 23s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. +1 unit 58m 53s hadoop-hdfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 86m 26s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830115/HDFS-10609.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e168785b063d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16849/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16849/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committing this based on Andrew's +1.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committing this based on Andrew's +1.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10489 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10489/)
        HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline (weichiu: rev 3ae652f82110a52bf239f3c1849b48981558eb19)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10489 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10489/ ) HDFS-10609 . Uncaught InvalidEncryptionKeyException during pipeline (weichiu: rev 3ae652f82110a52bf239f3c1849b48981558eb19) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockPoolTokenSecretManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committed the last patch to trunk, branch-2 and branch-2.8. Thanks very much Andrew Wang Xiao Chen for reviewing the patch!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committed the last patch to trunk, branch-2 and branch-2.8. Thanks very much Andrew Wang Xiao Chen for reviewing the patch!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10490 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10490/)
        Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException (weichiu: rev 8e06d865c4848f2ddd1a3ec4ee825e152d8e77c3)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10490 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10490/ ) Addendum patch for HDFS-10609 . Uncaught InvalidEncryptionKeyException (weichiu: rev 8e06d865c4848f2ddd1a3ec4ee825e152d8e77c3) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Reopen to attach a branch-2.7 patch.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Reopen to attach a branch-2.7 patch.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Branch-2.8 and above refactored DFSClient a lot. Attach a new patch for branch-2.7

        Show
        jojochuang Wei-Chiu Chuang added a comment - Branch-2.8 and above refactored DFSClient a lot. Attach a new patch for branch-2.7
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 31s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 12s branch-2.7 passed
        +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_111
        +1 checkstyle 0m 32s branch-2.7 passed
        +1 mvnsite 0m 59s branch-2.7 passed
        +1 mvneclipse 0m 15s branch-2.7 passed
        +1 findbugs 2m 58s branch-2.7 passed
        +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 40s branch-2.7 passed with JDK v1.7.0_111
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 55s the patch passed with JDK v1.8.0_101
        +1 javac 0m 55s the patch passed
        +1 compile 0m 57s the patch passed with JDK v1.7.0_111
        +1 javac 0m 57s the patch passed
        -0 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 584 unchanged - 5 fixed = 587 total (was 589)
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 2943 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 13s The patch 113 line(s) with tabs.
        +1 findbugs 3m 11s the patch passed
        +1 javadoc 0m 59s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_111
        -1 unit 44m 5s hadoop-hdfs in the patch failed with JDK v1.7.0_111.
        -1 asflicense 0m 19s The patch generated 3 ASF License warnings.
        139m 24s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.web.TestWebHDFS
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
        JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens
        JDK v1.7.0_111 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830907/HDFS-10609.branch-2.7.01.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ba5fc5893f86 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 7a2d602
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16926/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16926/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 12s branch-2.7 passed +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 32s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 15s branch-2.7 passed +1 findbugs 2m 58s branch-2.7 passed +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 40s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 52s the patch passed +1 compile 0m 55s the patch passed with JDK v1.8.0_101 +1 javac 0m 55s the patch passed +1 compile 0m 57s the patch passed with JDK v1.7.0_111 +1 javac 0m 57s the patch passed -0 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 584 unchanged - 5 fixed = 587 total (was 589) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 2943 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 13s The patch 113 line(s) with tabs. +1 findbugs 3m 11s the patch passed +1 javadoc 0m 59s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_111 -1 unit 44m 5s hadoop-hdfs in the patch failed with JDK v1.7.0_111. -1 asflicense 0m 19s The patch generated 3 ASF License warnings. 139m 24s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens JDK v1.7.0_111 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830907/HDFS-10609.branch-2.7.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ba5fc5893f86 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7a2d602 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16926/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16926/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16926/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Most code style warnings are unrelated. The test failure can not be reproduce on my local machine. Attach a slightly updated patch to trigger precommit again.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Most code style warnings are unrelated. The test failure can not be reproduce on my local machine. Attach a slightly updated patch to trigger precommit again.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 1s branch-2.7 passed
        +1 compile 1m 2s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 2s branch-2.7 passed with JDK v1.7.0_111
        +1 checkstyle 0m 31s branch-2.7 passed
        +1 mvnsite 0m 59s branch-2.7 passed
        +1 mvneclipse 0m 15s branch-2.7 passed
        +1 findbugs 2m 59s branch-2.7 passed
        +1 javadoc 0m 58s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 40s branch-2.7 passed with JDK v1.7.0_111
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 55s the patch passed with JDK v1.8.0_101
        +1 javac 0m 55s the patch passed
        +1 compile 0m 59s the patch passed with JDK v1.7.0_111
        +1 javac 0m 59s the patch passed
        -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 584 unchanged - 5 fixed = 587 total (was 589)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 3260 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 23s The patch 113 line(s) with tabs.
        +1 findbugs 3m 11s the patch passed
        +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 40s the patch passed with JDK v1.7.0_111
        -1 unit 47m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_111.
        -1 asflicense 0m 19s The patch generated 3 ASF License warnings.
        127m 37s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.web.TestWebHdfsTokens
          hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.web.TestWebHdfsTimeouts
        JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.balancer.TestBalancer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831172/HDFS-10609.branch-2.7.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 312a9c84dd6c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 7a2d602
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16956/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16956/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 1s branch-2.7 passed +1 compile 1m 2s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 2s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 31s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 15s branch-2.7 passed +1 findbugs 2m 59s branch-2.7 passed +1 javadoc 0m 58s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 40s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 53s the patch passed +1 compile 0m 55s the patch passed with JDK v1.8.0_101 +1 javac 0m 55s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_111 +1 javac 0m 59s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 584 unchanged - 5 fixed = 587 total (was 589) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 3260 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 23s The patch 113 line(s) with tabs. +1 findbugs 3m 11s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 40s the patch passed with JDK v1.7.0_111 -1 unit 47m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_111. -1 asflicense 0m 19s The patch generated 3 ASF License warnings. 127m 37s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.web.TestWebHdfsTimeouts JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831172/HDFS-10609.branch-2.7.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 312a9c84dd6c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7a2d602 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16956/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16956/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16956/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        The 2.7 patch passed in my local Yetus. Maybe it's something to do with the patch file name I guess? Upload a new patch file with a more "normalized" file name.

        Show
        jojochuang Wei-Chiu Chuang added a comment - The 2.7 patch passed in my local Yetus. Maybe it's something to do with the patch file name I guess? Upload a new patch file with a more "normalized" file name.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 15m 52s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 24s branch-2.7 passed
        +1 compile 1m 3s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 8s branch-2.7 passed with JDK v1.7.0_111
        +1 checkstyle 0m 34s branch-2.7 passed
        +1 mvnsite 1m 0s branch-2.7 passed
        +1 mvneclipse 0m 17s branch-2.7 passed
        +1 findbugs 3m 0s branch-2.7 passed
        +1 javadoc 1m 3s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_111
        +1 mvninstall 0m 57s the patch passed
        +1 compile 1m 4s the patch passed with JDK v1.8.0_101
        +1 javac 1m 4s the patch passed
        +1 compile 1m 2s the patch passed with JDK v1.7.0_111
        +1 javac 1m 2s the patch passed
        -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 583 unchanged - 5 fixed = 586 total (was 588)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 3597 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 27s The patch 113 line(s) with tabs.
        +1 findbugs 3m 8s the patch passed
        +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_111
        -1 unit 56m 20s hadoop-hdfs in the patch failed with JDK v1.7.0_111.
        -1 asflicense 0m 21s The patch generated 3 ASF License warnings.
        160m 35s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.TestRollingUpgrade
          hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
          hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
        JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestDNFencing
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.balancer.TestBalancer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue HDFS-10609
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831553/HDFS-10609.branch-2.7.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6e683acb329d 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / c08346e
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16998/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16998/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 52s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 24s branch-2.7 passed +1 compile 1m 3s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 8s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 34s branch-2.7 passed +1 mvnsite 1m 0s branch-2.7 passed +1 mvneclipse 0m 17s branch-2.7 passed +1 findbugs 3m 0s branch-2.7 passed +1 javadoc 1m 3s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 57s the patch passed +1 compile 1m 4s the patch passed with JDK v1.8.0_101 +1 javac 1m 4s the patch passed +1 compile 1m 2s the patch passed with JDK v1.7.0_111 +1 javac 1m 2s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 583 unchanged - 5 fixed = 586 total (was 588) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 3597 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 27s The patch 113 line(s) with tabs. +1 findbugs 3m 8s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_111 -1 unit 56m 20s hadoop-hdfs in the patch failed with JDK v1.7.0_111. -1 asflicense 0m 21s The patch generated 3 ASF License warnings. 160m 35s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-10609 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831553/HDFS-10609.branch-2.7.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6e683acb329d 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / c08346e Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16998/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16998/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16998/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        None of the tests failed in precommit failed in my local tree. All warnings are unrelated to this patch.

        Show
        jojochuang Wei-Chiu Chuang added a comment - None of the tests failed in precommit failed in my local tree. All warnings are unrelated to this patch.
        Hide
        andrew.wang Andrew Wang added a comment -

        +1 thanks Wei-Chiu

        Show
        andrew.wang Andrew Wang added a comment - +1 thanks Wei-Chiu
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committing this based on Andrew Wang's +1 and also the patch passed our internal testing. Thanks Andrew Wang!.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committing this based on Andrew Wang 's +1 and also the patch passed our internal testing. Thanks Andrew Wang !.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Hi Wei-Chiu Chuang, would you update CHANGES.txt when cherry-picking a commit to branch-2.7/2.6?
        HADOOP-13670 will fix this, so you don't need to update CHANGES.txt for now.

        Show
        ajisakaa Akira Ajisaka added a comment - Hi Wei-Chiu Chuang , would you update CHANGES.txt when cherry-picking a commit to branch-2.7/2.6? HADOOP-13670 will fix this, so you don't need to update CHANGES.txt for now.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Akira Ajisaka I'm so sorry about this. Will pay much more attention next time.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Akira Ajisaka I'm so sorry about this. Will pay much more attention next time.

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development