Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12336

Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha4
    • Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.3
    • Component/s: encryption, hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The fix proposed on HDFS-11197 didn't cover the scenario where the EZ deleted but still under a snapshot is not a direct child of the snapshottable directory.

      Here the code snippet proposed on HDFS-11197 that would avoid the error reported by hdfs crypto -listZones when a deleted EZ is still under a given snapshot:

            INode lastINode = null;
            if (inode.getParent() != null || inode.isRoot()) {
              INodesInPath iip = dir.getINodesInPath(pathName, DirOp.READ_LINK);
              lastINode = iip.getLastINode();
            }
            if (lastINode == null || lastINode.getId() != ezi.getINodeId()) {
              continue;
            }
      

      It will ignore EZs when it's a direct child of a snapshot, because its parent inode will be null, and it isn't the root inode. However, if the EZ is not directly under snapshottable directory, its parent will not be null, and it will pass this check, so it will fail further due absolute path required validation error.

      I would like to work on a fix that would also cover this scenario.

      1. HDFS-12336.001.patch
        6 kB
        Wellington Chevreuil
      2. HDFS-12336.002.patch
        10 kB
        Wellington Chevreuil
      3. HDFS-12336.003.patch
        10 kB
        Wellington Chevreuil
      4. HDFS-12336.004.patch
        10 kB
        Wellington Chevreuil
      5. HDFS-12336-branch-2.001.patch
        10 kB
        Wellington Chevreuil

        Issue Links

          Activity

          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Proposing initial patch with changes to also cover condition when EZ is not a direct child of snapshottable dir, together with tests to emulate such condition.

          Show
          wchevreuil Wellington Chevreuil added a comment - Proposing initial patch with changes to also cover condition when EZ is not a direct child of snapshottable dir, together with tests to emulate such condition.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 14m 35s trunk passed
          +1 compile 0m 52s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 0m 40s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 46s the patch passed
          +1 javac 0m 46s the patch passed
          +1 checkstyle 0m 33s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 39s the patch passed
                Other Tests
          -1 unit 68m 10s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          95m 13s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883120/HDFS-12336.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 9eecef036072 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 27ab5f7
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20804/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20804/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20804/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       trunk Compile Tests +1 mvninstall 14m 35s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 56s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 40s trunk passed       Patch Compile Tests +1 mvninstall 0m 48s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 68m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 95m 13s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883120/HDFS-12336.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 9eecef036072 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 27ab5f7 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/20804/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20804/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20804/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          I believe these test failures are not related. Had reviewed related code changes, it shouldn't affect these tests, and same tests are passing locally:

          Running org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.833 sec - in org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
          
          Results :
          
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
          
          Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
          Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.811 sec - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
          
          Results :
          
          Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
          
          Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
          Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.527 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
          
          Results :
          
          Tests run: 14, Failures: 0, Errors: 0, Skipped: 0
          
          Show
          wchevreuil Wellington Chevreuil added a comment - I believe these test failures are not related. Had reviewed related code changes, it shouldn't affect these tests, and same tests are passing locally: Running org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.833 sec - in org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy Results : Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.811 sec - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics Results : Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.527 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 Results : Tests run: 14, Failures: 0, Errors: 0, Skipped: 0
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Wellington Chevreuil for reporting the issue and providing a patch.

          It's not obvious to me what caused the error - IIUC getINodesInPath would throw the absolute path required, but that path is constructed from getFullPathName of the inode.

          Also tried the tests, but the TestEZM is highly mocked, and TestCryptoAdminCLI seems to pass with or without the fix. Since this part of code is mistaken a few times, starting from HDFS-7078, to the mentioned HDFS-11197, I wonder if it makes sense to do a real test in TestEncryptionZones? We could set up a combination of snapshots, and verify the result of listing in 1 test case. IMO this is more maintainable than mocking in TestEZM. We can start with the test cases, and pursue the correct fix from there.

          Show
          xiaochen Xiao Chen added a comment - Thanks Wellington Chevreuil for reporting the issue and providing a patch. It's not obvious to me what caused the error - IIUC getINodesInPath would throw the absolute path required, but that path is constructed from getFullPathName of the inode. Also tried the tests, but the TestEZM is highly mocked, and TestCryptoAdminCLI seems to pass with or without the fix. Since this part of code is mistaken a few times, starting from HDFS-7078 , to the mentioned HDFS-11197 , I wonder if it makes sense to do a real test in TestEncryptionZones? We could set up a combination of snapshots, and verify the result of listing in 1 test case. IMO this is more maintainable than mocking in TestEZM. We can start with the test cases, and pursue the correct fix from there.
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Hi Xiao Chen, thanks for reviewing this. I realised I made a mistake on the test added to TestCryptoAdminCLI, which was failing to capture the error when the fix is not applied.

          The problem manifests when an EZ is under a snapshottable directory. If a snapshot is taken, then the EZ (or a parent directory that contains the EZ) is moved to user's Trash folder, and later, it's completely removed from user's Trash, the EZ's inode path becomes somehow incomplete in the image. Here's an example of what inode.getFullPathName() returns in this case:

          INFO org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: >>>inode: dir10/dir11/ezz
          INFO org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: >>>inode: dir2/ez2
          

          I believe this happens because inode that got first moved to Trash, then deleted, ends up with no parent inode in the image, as if it was child of snaphots only.

          I agree it makes sense adding test to TestEncryptionZones. I'm including a patch with the additional test to TestEncryptionZones and fix to TestCryptoAdminCLI.

          Show
          wchevreuil Wellington Chevreuil added a comment - Hi Xiao Chen , thanks for reviewing this. I realised I made a mistake on the test added to TestCryptoAdminCLI, which was failing to capture the error when the fix is not applied. The problem manifests when an EZ is under a snapshottable directory. If a snapshot is taken, then the EZ (or a parent directory that contains the EZ) is moved to user's Trash folder, and later, it's completely removed from user's Trash, the EZ's inode path becomes somehow incomplete in the image. Here's an example of what inode.getFullPathName() returns in this case: INFO org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: >>>inode: dir10/dir11/ezz INFO org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: >>>inode: dir2/ez2 I believe this happens because inode that got first moved to Trash, then deleted, ends up with no parent inode in the image, as if it was child of snaphots only. I agree it makes sense adding test to TestEncryptionZones. I'm including a patch with the additional test to TestEncryptionZones and fix to TestCryptoAdminCLI.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s HDFS-12336 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883568/HDFS-12336.002.patch
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20850/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-12336 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883568/HDFS-12336.002.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20850/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks for revving and providing more details Wellington!

          It all makes sense to me, a few comments:

          • for the isValidEZPath check, can we check pathName for absolute path, instead of walking through the inode all the way to the root? INode.java currently has checkAbsolutePath which is exactly where the exception is thrown now. Perhaps we can extract the if condition to a helper method, and reuse it in our isValidEZPath check?
          • Suggest to use javadoc on the isValidEZPath method comment. Replacing the '/' with '/*' should do.
          • The added test looks good. I think we should call out why move to trash then delete is necessary (instead of fsWrapper.delete): to create a inodereference for the test.
          • Instead of 2 assertZonePresent, it feels to me we could list the zones once, then check things while going through the iterator.
          • Similarly for the last assert, maybe we can verify from listing directly, rather than the current way of catching assertion error on a method named assertZonePresent and assert the message.
          Show
          xiaochen Xiao Chen added a comment - Thanks for revving and providing more details Wellington! It all makes sense to me, a few comments: for the isValidEZPath check, can we check pathName for absolute path, instead of walking through the inode all the way to the root? INode.java currently has checkAbsolutePath which is exactly where the exception is thrown now. Perhaps we can extract the if condition to a helper method, and reuse it in our isValidEZPath check? Suggest to use javadoc on the isValidEZPath method comment. Replacing the '/ ' with '/ *' should do. The added test looks good. I think we should call out why move to trash then delete is necessary (instead of fsWrapper.delete): to create a inodereference for the test. Instead of 2 assertZonePresent , it feels to me we could list the zones once, then check things while going through the iterator. Similarly for the last assert, maybe we can verify from listing directly, rather than the current way of catching assertion error on a method named assertZonePresent and assert the message.
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Thanks for the suggestions Xiao Chen! Indeed, checking the absolute path is much simpler. I thought of defining a new helper method, isValidAbsolutePath on INode class that does the check previously done on checkAbsolutePath, then using this new one on EncryptionZoneManager.listEncryptionZones to determine if the inode should be filtered or not.

          Also had applied the suggestions to EncryptionZonesTest, had added javadoc explaining why we need to move to Trash first, instead of simply deleting it, and also removed the calls to assertZonePresent.

          I also noticed latest patch was out of sync with trunk. I had rebased it to latest trunk version and had resolved some conflicts with HDFS-10899, which has doe a large refactoring in EncryptionZoneManager. These scenario was still failing, though.

          Show
          wchevreuil Wellington Chevreuil added a comment - Thanks for the suggestions Xiao Chen ! Indeed, checking the absolute path is much simpler. I thought of defining a new helper method, isValidAbsolutePath on INode class that does the check previously done on checkAbsolutePath , then using this new one on EncryptionZoneManager.listEncryptionZones to determine if the inode should be filtered or not. Also had applied the suggestions to EncryptionZonesTest , had added javadoc explaining why we need to move to Trash first, instead of simply deleting it, and also removed the calls to assertZonePresent. I also noticed latest patch was out of sync with trunk. I had rebased it to latest trunk version and had resolved some conflicts with HDFS-10899 , which has doe a large refactoring in EncryptionZoneManager. These scenario was still failing, though.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 17m 41s trunk passed
          +1 compile 1m 0s trunk passed
          +1 checkstyle 0m 46s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 findbugs 2m 13s trunk passed
          +1 javadoc 0m 53s trunk passed
                Patch Compile Tests
          +1 mvninstall 1m 12s the patch passed
          +1 compile 1m 6s the patch passed
          +1 javac 1m 6s the patch passed
          -0 checkstyle 0m 45s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 96 unchanged - 0 fixed = 97 total (was 96)
          +1 mvnsite 1m 17s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 0m 38s the patch passed
                Other Tests
          -1 unit 68m 10s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          101m 4s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.TestDFSClientRetries



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883763/HDFS-12336.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 503af96149ee 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3a4e861
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20863/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20863/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20863/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20863/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 41s trunk passed +1 compile 1m 0s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 15s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 0m 53s trunk passed       Patch Compile Tests +1 mvninstall 1m 12s the patch passed +1 compile 1m 6s the patch passed +1 javac 1m 6s the patch passed -0 checkstyle 0m 45s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 96 unchanged - 0 fixed = 97 total (was 96) +1 mvnsite 1m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 38s the patch passed       Other Tests -1 unit 68m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 101m 4s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestDFSClientRetries Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12883763/HDFS-12336.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 503af96149ee 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3a4e861 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20863/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/20863/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20863/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20863/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment - - edited

          Thanks for the new patch Wellington, looks pretty good to me. I think we're really close. (Yea, HDFS-10899 was me, sorry for the rebase )

          Comments, mostly nits:

          • I think we should pass in zonePath, instead of inode.getFullPathName to isValidAbsolutePath. The latter is more expensive, and later getINodesInPath is resolving the former.
          • isValidAbsolutePath could be package private, no public necessary
          • Nit: I think we can leave out the outer () in return (path != null && path.startsWith(Path.SEPARATOR));
          • Nit: Test could use assertNotEquals instead of assertFalse(x.equals(z)).
          • Please fix checkstyle while you're at it. (I can't explain why, but we really love the '.' at the end of the first sentence)
          Show
          xiaochen Xiao Chen added a comment - - edited Thanks for the new patch Wellington, looks pretty good to me. I think we're really close. (Yea, HDFS-10899 was me, sorry for the rebase ) Comments, mostly nits: I think we should pass in zonePath , instead of inode.getFullPathName to isValidAbsolutePath . The latter is more expensive, and later getINodesInPath is resolving the former. isValidAbsolutePath could be package private, no public necessary Nit: I think we can leave out the outer () in return (path != null && path.startsWith(Path.SEPARATOR)); Nit: Test could use assertNotEquals instead of assertFalse(x.equals(z)) . Please fix checkstyle while you're at it. (I can't explain why, but we really love the '.' at the end of the first sentence)
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Thanks for the latest suggestions Xiao Chen! All make sense to me, am attaching a new patch with those applied.

          If these are ok, let me know if you want me to create a patch for branch-2 as well.

          Show
          wchevreuil Wellington Chevreuil added a comment - Thanks for the latest suggestions Xiao Chen ! All make sense to me, am attaching a new patch with those applied. If these are ok, let me know if you want me to create a patch for branch-2 as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 14m 43s trunk passed
          +1 compile 0m 53s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 1m 1s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 43s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 52s the patch passed
          +1 javac 0m 52s the patch passed
          +1 checkstyle 0m 37s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 56s the patch passed
          +1 javadoc 0m 41s the patch passed
                Other Tests
          -1 unit 93m 11s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          120m 41s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestPread
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
            hadoop.hdfs.TestReconstructStripedFile
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884233/HDFS-12336.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux a73aa0d78ec3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7ddc7aa
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20907/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20907/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20907/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 14m 43s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 1m 1s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 43s trunk passed       Patch Compile Tests +1 mvninstall 0m 53s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 41s the patch passed       Other Tests -1 unit 93m 11s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 120m 41s Reason Tests Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestPread   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210   hadoop.hdfs.TestReconstructStripedFile Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884233/HDFS-12336.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux a73aa0d78ec3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7ddc7aa Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/20907/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20907/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20907/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Patch 4 LGTM, thanks Wellington Chevreuil.

          Failed tests look unrelated, but triggered another run just in case. Also verified added tests fail-before, pass-after.

          Show
          xiaochen Xiao Chen added a comment - Patch 4 LGTM, thanks Wellington Chevreuil . Failed tests look unrelated, but triggered another run just in case. Also verified added tests fail-before, pass-after.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 38s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 15m 15s trunk passed
          +1 compile 0m 55s trunk passed
          +1 checkstyle 0m 40s trunk passed
          +1 mvnsite 1m 3s trunk passed
          +1 findbugs 1m 50s trunk passed
          +1 javadoc 0m 44s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 57s the patch passed
          +1 compile 0m 52s the patch passed
          +1 javac 0m 52s the patch passed
          +1 checkstyle 0m 38s the patch passed
          +1 mvnsite 0m 57s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 43s the patch passed
                Other Tests
          -1 unit 116m 16s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          145m 29s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestLeaseRecovery
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
            hadoop.hdfs.TestLeaseRecoveryStriped
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.TestFileAppendRestart
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.TestEncryptedTransfer
          Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile
            org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884233/HDFS-12336.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux e39dcd368d6e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 63fc1b0
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20911/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20911/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20911/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 38s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       trunk Compile Tests +1 mvninstall 15m 15s trunk passed +1 compile 0m 55s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 3s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 0m 44s trunk passed       Patch Compile Tests +1 mvninstall 0m 57s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed +1 checkstyle 0m 38s the patch passed +1 mvnsite 0m 57s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 43s the patch passed       Other Tests -1 unit 116m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 145m 29s Reason Tests Failed junit tests hadoop.hdfs.TestLeaseRecovery   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestEncryptedTransfer Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile   org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884233/HDFS-12336.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux e39dcd368d6e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 63fc1b0 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/20911/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20911/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20911/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Compared failed tests side-by-side, the only intersections are TestLeaseRecoveryStriped and TestWriteReadStripedFile, tracked by HDFS-12360 and not related to the changes here.

          +1 to patch 4, committing this.

          Show
          xiaochen Xiao Chen added a comment - Compared failed tests side-by-side, the only intersections are TestLeaseRecoveryStriped and TestWriteReadStripedFile, tracked by HDFS-12360 and not related to the changes here. +1 to patch 4, committing this.
          Hide
          xiaochen Xiao Chen added a comment -

          Committed this to trunk.

          Wellington Chevreuil, you're right, branch-2 patch needed.

          Show
          xiaochen Xiao Chen added a comment - Committed this to trunk. Wellington Chevreuil , you're right, branch-2 patch needed.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12269 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12269/)
          HDFS-12336. Listing encryption zones still fails when deleted EZ is not (xiao: rev f3661fd08e03440d02452b46ba3ae3cff2b75ba7)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEncryptionZoneManager.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionZoneManager.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCryptoConf.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12269 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12269/ ) HDFS-12336 . Listing encryption zones still fails when deleted EZ is not (xiao: rev f3661fd08e03440d02452b46ba3ae3cff2b75ba7) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEncryptionZoneManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionZoneManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCryptoConf.xml
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Thanks a lot Xiao Chen for the help here. Attached a patch for branch-2.

          Show
          wchevreuil Wellington Chevreuil added a comment - Thanks a lot Xiao Chen for the help here. Attached a patch for branch-2.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
                branch-2 Compile Tests
          +1 mvninstall 6m 54s branch-2 passed
          +1 compile 0m 42s branch-2 passed with JDK v1.8.0_144
          +1 compile 0m 43s branch-2 passed with JDK v1.7.0_131
          +1 checkstyle 0m 29s branch-2 passed
          +1 mvnsite 0m 51s branch-2 passed
          +1 findbugs 2m 1s branch-2 passed
          +1 javadoc 0m 39s branch-2 passed with JDK v1.8.0_144
          +1 javadoc 0m 59s branch-2 passed with JDK v1.7.0_131
                Patch Compile Tests
          +1 mvninstall 0m 44s the patch passed
          +1 compile 0m 37s the patch passed with JDK v1.8.0_144
          +1 javac 0m 37s the patch passed
          +1 compile 0m 41s the patch passed with JDK v1.7.0_131
          +1 javac 0m 41s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 0m 49s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 12s the patch passed
          +1 javadoc 0m 35s the patch passed with JDK v1.8.0_144
          +1 javadoc 1m 1s the patch passed with JDK v1.7.0_131
                Other Tests
          -1 unit 54m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
          -1 asflicense 0m 22s The patch generated 2 ASF License warnings.
          140m 10s



          Reason Tests
          JDK v1.8.0_144 Failed junit tests hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.TestEncryptedTransfer
          JDK v1.8.0_144 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
          JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestEncryptedTransfer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue HDFS-12336
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884449/HDFS-12336-branch-2.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 7ceeb7b15bb0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 877d96b
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/20922/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20922/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/20922/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20922/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.       branch-2 Compile Tests +1 mvninstall 6m 54s branch-2 passed +1 compile 0m 42s branch-2 passed with JDK v1.8.0_144 +1 compile 0m 43s branch-2 passed with JDK v1.7.0_131 +1 checkstyle 0m 29s branch-2 passed +1 mvnsite 0m 51s branch-2 passed +1 findbugs 2m 1s branch-2 passed +1 javadoc 0m 39s branch-2 passed with JDK v1.8.0_144 +1 javadoc 0m 59s branch-2 passed with JDK v1.7.0_131       Patch Compile Tests +1 mvninstall 0m 44s the patch passed +1 compile 0m 37s the patch passed with JDK v1.8.0_144 +1 javac 0m 37s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_131 +1 javac 0m 41s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 12s the patch passed +1 javadoc 0m 35s the patch passed with JDK v1.8.0_144 +1 javadoc 1m 1s the patch passed with JDK v1.7.0_131       Other Tests -1 unit 54m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_131. -1 asflicense 0m 22s The patch generated 2 ASF License warnings. 140m 10s Reason Tests JDK v1.8.0_144 Failed junit tests hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.TestEncryptedTransfer JDK v1.8.0_144 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestEncryptedTransfer Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue HDFS-12336 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884449/HDFS-12336-branch-2.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 7ceeb7b15bb0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 877d96b Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/20922/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20922/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/20922/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20922/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          wchevreuil Wellington Chevreuil added a comment -

          Had a look at last build failed tests, don't think any of those relate to the code changes. Those are also passing on my local build.

          Show
          wchevreuil Wellington Chevreuil added a comment - Had a look at last build failed tests, don't think any of those relate to the code changes. Those are also passing on my local build.
          Hide
          xiaochen Xiao Chen added a comment -

          Committed to branch-2 and branch-2.8. Thank you, Wellington Chevreuil!

          Show
          xiaochen Xiao Chen added a comment - Committed to branch-2 and branch-2.8. Thank you, Wellington Chevreuil !

            People

            • Assignee:
              wchevreuil Wellington Chevreuil
              Reporter:
              wchevreuil Wellington Chevreuil
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development