Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10489

Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.4
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      The configuration dfs.encryption.key.provider.uri is deprecated. To configure key provider in HDFS, please use hadoop.security.key.provider.path.

      Description

      When working on HADOOP-13155, we discussed and concluded that we should use the common config key for key provider uri.

      We can depreate the dfs. key for 3.0.0.

      1. HDFS-10489.01.patch
        12 kB
        Xiao Chen
      2. HDFS-10489.02.patch
        12 kB
        Xiao Chen
      3. HDFS-10489.03.patch
        14 kB
        Xiao Chen
      4. HDFS-10489.04.patch
        16 kB
        Xiao Chen
      5. HDFS-10489.05.patch
        33 kB
        Xiao Chen
      6. HDFS-10489.06.patch
        33 kB
        Xiao Chen

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        I'd like to get this done by alpha2.
        Attaching a patch 1 to remove the dfs key and use the hadoop key for 3.0. I didn't find a canonical hadoop jira for doing this, so please advise if anything not right. (I found HDFS-5570 so went similar approach).

        We should have another patch for branch-2 to mark the dfs config as @Deprecated.

        I kept the DFSConfigKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI so less places need to be changed. But maybe we should really remove it? What's your thought Andrew Wang? Thanks!

        Show
        xiaochen Xiao Chen added a comment - I'd like to get this done by alpha2. Attaching a patch 1 to remove the dfs key and use the hadoop key for 3.0. I didn't find a canonical hadoop jira for doing this, so please advise if anything not right. (I found HDFS-5570 so went similar approach). We should have another patch for branch-2 to mark the dfs config as @Deprecated . I kept the DFSConfigKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI so less places need to be changed. But maybe we should really remove it? What's your thought Andrew Wang ? Thanks!
        Hide
        andrew.wang Andrew Wang added a comment -

        Deprecation typically means adding a DeprecationDelta so that it transitions over to the new, preferred key. In cases like this, there might not be much benefit from outright removing the key compared to just deprecating it since the config value is the same.

        Show
        andrew.wang Andrew Wang added a comment - Deprecation typically means adding a DeprecationDelta so that it transitions over to the new, preferred key. In cases like this, there might not be much benefit from outright removing the key compared to just deprecating it since the config value is the same.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for the explanation, so this jira will deprecate in 3.x, and remove it in 4.x, right? I was under the wrong impression that this will be done for 2.x / 3.x, respectively.....
        Will update a new patch soon. Thanks Andrew.

        Show
        xiaochen Xiao Chen added a comment - Thanks for the explanation, so this jira will deprecate in 3.x, and remove it in 4.x, right? I was under the wrong impression that this will be done for 2.x / 3.x, respectively..... Will update a new patch soon. Thanks Andrew.
        Hide
        andrew.wang Andrew Wang added a comment -

        We can certainly deprecate 2.x and remove for 3.x if we wish, but I'm wondering if there's value to removing over just deprecating.

        Typically we remove in situations like:

        1. Implementation has changed such that there's no good way of translating the old key -> new key.
        2. If there's a maintenance burden to keeping the old key around
        3. It's confusing for users. We should try to minimize surprise.

        I believe the first two do not apply here.

        Regarding users, let's say a 2.x client is using the dfs.encryption.key.provider.uri config key. Then, they upgrade to 3.x. If we remove the dfs key, they'll get an error that hadoop.security... is not configured, and need to change that key name over.

        Alternatively, if we add a DeprecationDelta, the old key will be forwarded to the new one, a warning will go into their log file, but the client will still work. I feel this is a nicer user experience.

        Show
        andrew.wang Andrew Wang added a comment - We can certainly deprecate 2.x and remove for 3.x if we wish, but I'm wondering if there's value to removing over just deprecating. Typically we remove in situations like: Implementation has changed such that there's no good way of translating the old key -> new key. If there's a maintenance burden to keeping the old key around It's confusing for users. We should try to minimize surprise. I believe the first two do not apply here. Regarding users, let's say a 2.x client is using the dfs.encryption.key.provider.uri config key. Then, they upgrade to 3.x. If we remove the dfs key, they'll get an error that hadoop.security... is not configured, and need to change that key name over. Alternatively, if we add a DeprecationDelta, the old key will be forwarded to the new one, a warning will go into their log file, but the client will still work. I feel this is a nicer user experience.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for the input, Andrew! It totally makes sense to me, attached a patch 2 to deprecate it.

        Another thing here is, I added a fall back logic on current usage of the dfs key, to also try the hadoop key before fail out. This is incompatible, but would make the 3.x to 4.x upgrade friendlier, since people can choose to use the hadoop key in 3, and theoretically have a transparent upgrade to 4.

        Show
        xiaochen Xiao Chen added a comment - Thanks for the input, Andrew! It totally makes sense to me, attached a patch 2 to deprecate it. Another thing here is, I added a fall back logic on current usage of the dfs key, to also try the hadoop key before fail out. This is incompatible, but would make the 3.x to 4.x upgrade friendlier, since people can choose to use the hadoop key in 3, and theoretically have a transparent upgrade to 4.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 1m 24s Maven dependency ordering for branch
        +1 mvninstall 6m 49s trunk passed
        +1 compile 6m 50s trunk passed
        +1 checkstyle 1m 32s trunk passed
        +1 mvnsite 2m 40s trunk passed
        +1 mvneclipse 0m 50s trunk passed
        +1 findbugs 4m 45s trunk passed
        +1 javadoc 2m 13s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 13s the patch passed
        +1 compile 6m 46s the patch passed
        -1 javac 6m 46s root generated 3 new + 708 unchanged - 0 fixed = 711 total (was 708)
        -0 checkstyle 1m 31s root: The patch generated 1 new + 556 unchanged - 0 fixed = 557 total (was 556)
        +1 mvnsite 2m 37s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 3s The patch has no ill-formed XML file.
        +1 findbugs 5m 20s the patch passed
        +1 javadoc 2m 11s the patch passed
        +1 unit 7m 22s hadoop-common in the patch passed.
        +1 unit 2m 6s hadoop-kms in the patch passed.
        +1 unit 0m 57s hadoop-hdfs-client in the patch passed.
        -1 unit 58m 19s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        119m 20s



        Reason Tests
        Failed junit tests hadoop.tools.TestHdfsConfigFields
          hadoop.hdfs.TestFileCreationDelete
          hadoop.hdfs.TestRenameWhileOpen



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828566/HDFS-10489.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux e123cc89f692 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2a8f55a
        Default Java 1.8.0_101
        findbugs v3.0.0
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/diff-compile-javac-root.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16750/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16750/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 1m 24s Maven dependency ordering for branch +1 mvninstall 6m 49s trunk passed +1 compile 6m 50s trunk passed +1 checkstyle 1m 32s trunk passed +1 mvnsite 2m 40s trunk passed +1 mvneclipse 0m 50s trunk passed +1 findbugs 4m 45s trunk passed +1 javadoc 2m 13s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 13s the patch passed +1 compile 6m 46s the patch passed -1 javac 6m 46s root generated 3 new + 708 unchanged - 0 fixed = 711 total (was 708) -0 checkstyle 1m 31s root: The patch generated 1 new + 556 unchanged - 0 fixed = 557 total (was 556) +1 mvnsite 2m 37s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 5m 20s the patch passed +1 javadoc 2m 11s the patch passed +1 unit 7m 22s hadoop-common in the patch passed. +1 unit 2m 6s hadoop-kms in the patch passed. +1 unit 0m 57s hadoop-hdfs-client in the patch passed. -1 unit 58m 19s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 119m 20s Reason Tests Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestRenameWhileOpen Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828566/HDFS-10489.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux e123cc89f692 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a8f55a Default Java 1.8.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16750/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16750/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16750/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Had an offline chat with Andrew, turns out Configuration#handleDeprecation elegantly handles this. Updated patch 3 to use this.
        Sorry I should have looked deeper. Thanks Andrew!

        Show
        xiaochen Xiao Chen added a comment - Had an offline chat with Andrew, turns out Configuration#handleDeprecation elegantly handles this. Updated patch 3 to use this. Sorry I should have looked deeper. Thanks Andrew!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 18s Maven dependency ordering for branch
        +1 mvninstall 8m 8s trunk passed
        +1 compile 8m 9s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 3m 3s trunk passed
        +1 mvneclipse 0m 54s trunk passed
        +1 findbugs 5m 15s trunk passed
        +1 javadoc 2m 13s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 26s the patch passed
        +1 compile 6m 45s the patch passed
        +1 javac 6m 45s the patch passed
        +1 checkstyle 1m 30s the patch passed
        +1 mvnsite 2m 35s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 3s The patch has no ill-formed XML file.
        +1 findbugs 5m 12s the patch passed
        -1 javadoc 0m 20s hadoop-hdfs-project_hadoop-hdfs-client generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)
        +1 unit 7m 31s hadoop-common in the patch passed.
        +1 unit 2m 6s hadoop-kms in the patch passed.
        +1 unit 0m 56s hadoop-hdfs-client in the patch passed.
        -1 unit 63m 44s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        127m 21s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.TestReconstructStripedFile
          hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          hadoop.hdfs.TestEncryptionZonesWithKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828723/HDFS-10489.03.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 4aa56a1981ee 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / fcbac00
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/16756/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16756/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16756/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16756/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 8m 8s trunk passed +1 compile 8m 9s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 3m 3s trunk passed +1 mvneclipse 0m 54s trunk passed +1 findbugs 5m 15s trunk passed +1 javadoc 2m 13s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 26s the patch passed +1 compile 6m 45s the patch passed +1 javac 6m 45s the patch passed +1 checkstyle 1m 30s the patch passed +1 mvnsite 2m 35s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 5m 12s the patch passed -1 javadoc 0m 20s hadoop-hdfs-project_hadoop-hdfs-client generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) +1 unit 7m 31s hadoop-common in the patch passed. +1 unit 2m 6s hadoop-kms in the patch passed. +1 unit 0m 56s hadoop-hdfs-client in the patch passed. -1 unit 63m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 127m 21s Reason Tests Failed junit tests hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.TestEncryptionZonesWithKMS Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828723/HDFS-10489.03.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 4aa56a1981ee 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fcbac00 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/16756/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16756/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16756/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16756/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Patch 4 to make pre-commit happy

        Show
        xiaochen Xiao Chen added a comment - Patch 4 to make pre-commit happy
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for revving, some review comments:

        • Any reason we don't purge all usages of DFSConfigKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI from the code base? We should try and set a good example in our tests.
        • Same for HdfsClientConfigKeys.DeprecatedKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI and its usages, we can move over to the new property.
        • hdfs-default.xml, I think we can just delete the old config key and depend on the warning message for users who are still setting the old key. Can add a release note to this JIRA to help raise awareness too, people don't typically diff the docs when upgrading between releases.
        • In the docs, I don't think we need the new NOTE for the same reason as previous.

        These are pretty small things, so close to +1 overall. Thanks for pushing on this Xiao!

        Show
        andrew.wang Andrew Wang added a comment - Thanks for revving, some review comments: Any reason we don't purge all usages of DFSConfigKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI from the code base? We should try and set a good example in our tests. Same for HdfsClientConfigKeys.DeprecatedKeys#DFS_ENCRYPTION_KEY_PROVIDER_URI and its usages, we can move over to the new property. hdfs-default.xml, I think we can just delete the old config key and depend on the warning message for users who are still setting the old key. Can add a release note to this JIRA to help raise awareness too, people don't typically diff the docs when upgrading between releases. In the docs, I don't think we need the new NOTE for the same reason as previous. These are pretty small things, so close to +1 overall. Thanks for pushing on this Xiao!
        Hide
        xiaochen Xiao Chen added a comment -

        Thank you for the quick turnaround Andrew, and great guidance.
        Patch 5 to address all comments.

        Show
        xiaochen Xiao Chen added a comment - Thank you for the quick turnaround Andrew, and great guidance. Patch 5 to address all comments.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 21s Maven dependency ordering for branch
        +1 mvninstall 6m 46s trunk passed
        +1 compile 7m 9s trunk passed
        +1 checkstyle 1m 40s trunk passed
        +1 mvnsite 2m 55s trunk passed
        +1 mvneclipse 0m 53s trunk passed
        +1 findbugs 5m 32s trunk passed
        +1 javadoc 2m 23s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 2m 39s the patch passed
        +1 compile 7m 37s the patch passed
        +1 javac 7m 37s the patch passed
        -0 checkstyle 1m 34s root: The patch generated 1 new + 603 unchanged - 0 fixed = 604 total (was 603)
        +1 mvnsite 2m 55s the patch passed
        +1 mvneclipse 0m 54s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 3s The patch has no ill-formed XML file.
        +1 findbugs 5m 43s the patch passed
        -1 javadoc 0m 20s hadoop-hdfs-project_hadoop-hdfs-client generated 3 new + 1 unchanged - 0 fixed = 4 total (was 1)
        +1 unit 9m 23s hadoop-common in the patch passed.
        +1 unit 2m 7s hadoop-kms in the patch passed.
        +1 unit 1m 7s hadoop-hdfs-client in the patch passed.
        -1 unit 66m 7s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        133m 11s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSClientRetries



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828771/HDFS-10489.04.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 6c334b8555c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ec3ea18
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16763/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16763/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 21s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 7m 9s trunk passed +1 checkstyle 1m 40s trunk passed +1 mvnsite 2m 55s trunk passed +1 mvneclipse 0m 53s trunk passed +1 findbugs 5m 32s trunk passed +1 javadoc 2m 23s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 39s the patch passed +1 compile 7m 37s the patch passed +1 javac 7m 37s the patch passed -0 checkstyle 1m 34s root: The patch generated 1 new + 603 unchanged - 0 fixed = 604 total (was 603) +1 mvnsite 2m 55s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 5m 43s the patch passed -1 javadoc 0m 20s hadoop-hdfs-project_hadoop-hdfs-client generated 3 new + 1 unchanged - 0 fixed = 4 total (was 1) +1 unit 9m 23s hadoop-common in the patch passed. +1 unit 2m 7s hadoop-kms in the patch passed. +1 unit 1m 7s hadoop-hdfs-client in the patch passed. -1 unit 66m 7s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 133m 11s Reason Tests Failed junit tests hadoop.hdfs.TestDFSClientRetries Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828771/HDFS-10489.04.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 6c334b8555c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ec3ea18 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16763/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16763/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 12 new or modified test files.
        0 mvndep 1m 29s Maven dependency ordering for branch
        +1 mvninstall 8m 14s trunk passed
        +1 compile 8m 33s trunk passed
        +1 checkstyle 2m 0s trunk passed
        +1 mvnsite 4m 11s trunk passed
        +1 mvneclipse 1m 41s trunk passed
        +1 findbugs 6m 31s trunk passed
        +1 javadoc 2m 58s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 3m 41s the patch passed
        +1 compile 8m 53s the patch passed
        +1 javac 8m 53s the patch passed
        -0 checkstyle 1m 56s root: The patch generated 1 new + 808 unchanged - 6 fixed = 809 total (was 814)
        +1 mvnsite 4m 4s the patch passed
        +1 mvneclipse 1m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 3s The patch has no ill-formed XML file.
        +1 findbugs 7m 24s the patch passed
        +1 javadoc 3m 9s the patch passed
        -1 unit 22m 37s hadoop-common in the patch failed.
        +1 unit 2m 22s hadoop-kms in the patch passed.
        +1 unit 0m 59s hadoop-hdfs-client in the patch passed.
        -1 unit 69m 4s hadoop-hdfs in the patch failed.
        +1 unit 3m 25s hadoop-hdfs-httpfs in the patch passed.
        +1 unit 1m 48s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        168m 59s



        Reason Tests
        Failed junit tests hadoop.tools.TestHdfsConfigFields
          hadoop.hdfs.server.namenode.TestCacheDirectives
        Timed out junit tests org.apache.hadoop.fs.TestFsShellReturnCode
          org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828781/HDFS-10489.05.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 1ed5621fa455 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ec3ea18
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16766/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16766/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 12 new or modified test files. 0 mvndep 1m 29s Maven dependency ordering for branch +1 mvninstall 8m 14s trunk passed +1 compile 8m 33s trunk passed +1 checkstyle 2m 0s trunk passed +1 mvnsite 4m 11s trunk passed +1 mvneclipse 1m 41s trunk passed +1 findbugs 6m 31s trunk passed +1 javadoc 2m 58s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 3m 41s the patch passed +1 compile 8m 53s the patch passed +1 javac 8m 53s the patch passed -0 checkstyle 1m 56s root: The patch generated 1 new + 808 unchanged - 6 fixed = 809 total (was 814) +1 mvnsite 4m 4s the patch passed +1 mvneclipse 1m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 7m 24s the patch passed +1 javadoc 3m 9s the patch passed -1 unit 22m 37s hadoop-common in the patch failed. +1 unit 2m 22s hadoop-kms in the patch passed. +1 unit 0m 59s hadoop-hdfs-client in the patch passed. -1 unit 69m 4s hadoop-hdfs in the patch failed. +1 unit 3m 25s hadoop-hdfs-httpfs in the patch passed. +1 unit 1m 48s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 168m 59s Reason Tests Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.namenode.TestCacheDirectives Timed out junit tests org.apache.hadoop.fs.TestFsShellReturnCode   org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828781/HDFS-10489.05.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 1ed5621fa455 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ec3ea18 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16766/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16766/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16766/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Reattached patch 5 without the 1 used import.

        Show
        xiaochen Xiao Chen added a comment - Reattached patch 5 without the 1 used import.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 12 new or modified test files.
        0 mvndep 1m 22s Maven dependency ordering for branch
        +1 mvninstall 7m 28s trunk passed
        +1 compile 7m 44s trunk passed
        +1 checkstyle 1m 40s trunk passed
        +1 mvnsite 3m 38s trunk passed
        +1 mvneclipse 1m 15s trunk passed
        +1 findbugs 6m 7s trunk passed
        +1 javadoc 2m 46s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 3m 9s the patch passed
        +1 compile 7m 38s the patch passed
        +1 javac 7m 38s the patch passed
        +1 checkstyle 1m 38s root: The patch generated 0 new + 808 unchanged - 6 fixed = 808 total (was 814)
        +1 mvnsite 3m 35s the patch passed
        +1 mvneclipse 1m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 6m 55s the patch passed
        +1 javadoc 2m 45s the patch passed
        +1 unit 8m 1s hadoop-common in the patch passed.
        +1 unit 2m 7s hadoop-kms in the patch passed.
        +1 unit 0m 59s hadoop-hdfs-client in the patch passed.
        -1 unit 61m 32s hadoop-hdfs in the patch failed.
        +1 unit 3m 36s hadoop-hdfs-httpfs in the patch passed.
        +1 unit 1m 49s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        139m 21s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestFileCreationDelete
          hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
          hadoop.tools.TestHdfsConfigFields



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828800/HDFS-10489.05.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 2c7cd730e96a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ec3ea18
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16768/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16768/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16768/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 12 new or modified test files. 0 mvndep 1m 22s Maven dependency ordering for branch +1 mvninstall 7m 28s trunk passed +1 compile 7m 44s trunk passed +1 checkstyle 1m 40s trunk passed +1 mvnsite 3m 38s trunk passed +1 mvneclipse 1m 15s trunk passed +1 findbugs 6m 7s trunk passed +1 javadoc 2m 46s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 3m 9s the patch passed +1 compile 7m 38s the patch passed +1 javac 7m 38s the patch passed +1 checkstyle 1m 38s root: The patch generated 0 new + 808 unchanged - 6 fixed = 808 total (was 814) +1 mvnsite 3m 35s the patch passed +1 mvneclipse 1m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 6m 55s the patch passed +1 javadoc 2m 45s the patch passed +1 unit 8m 1s hadoop-common in the patch passed. +1 unit 2m 7s hadoop-kms in the patch passed. +1 unit 0m 59s hadoop-hdfs-client in the patch passed. -1 unit 61m 32s hadoop-hdfs in the patch failed. +1 unit 3m 36s hadoop-hdfs-httpfs in the patch passed. +1 unit 1m 49s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 139m 21s Reason Tests Failed junit tests hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.tools.TestHdfsConfigFields Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828800/HDFS-10489.05.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 2c7cd730e96a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ec3ea18 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16768/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16768/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16768/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        I think we can also get rid of the DFS_ENCRYPTION_KEY_PROVIDER_URI in DFSConfigKeys and HdfsClientConfigKeys, they are both private classes.

        +1 pending though, nice work here.

        Show
        andrew.wang Andrew Wang added a comment - I think we can also get rid of the DFS_ENCRYPTION_KEY_PROVIDER_URI in DFSConfigKeys and HdfsClientConfigKeys, they are both private classes. +1 pending though, nice work here.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Andrew, good point! As long as DD let the old key work, we really should remove all references as a good example.
        Patch 6 to make this cleaner.

        Show
        xiaochen Xiao Chen added a comment - Thanks Andrew, good point! As long as DD let the old key work, we really should remove all references as a good example. Patch 6 to make this cleaner.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 12 new or modified test files.
        0 mvndep 1m 34s Maven dependency ordering for branch
        +1 mvninstall 7m 22s trunk passed
        +1 compile 6m 46s trunk passed
        +1 checkstyle 1m 34s trunk passed
        +1 mvnsite 3m 21s trunk passed
        +1 mvneclipse 1m 14s trunk passed
        +1 findbugs 5m 28s trunk passed
        +1 javadoc 2m 38s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 3m 29s the patch passed
        +1 compile 8m 22s the patch passed
        +1 javac 8m 22s the patch passed
        +1 checkstyle 1m 39s root: The patch generated 0 new + 808 unchanged - 6 fixed = 808 total (was 814)
        +1 mvnsite 3m 54s the patch passed
        +1 mvneclipse 1m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 7m 25s the patch passed
        +1 javadoc 2m 57s the patch passed
        +1 unit 8m 31s hadoop-common in the patch passed.
        +1 unit 2m 11s hadoop-kms in the patch passed.
        +1 unit 1m 3s hadoop-hdfs-client in the patch passed.
        -1 unit 64m 37s hadoop-hdfs in the patch failed.
        +1 unit 3m 40s hadoop-hdfs-httpfs in the patch passed.
        +1 unit 1m 50s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        143m 17s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10489
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828904/HDFS-10489.06.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 24672b0bd19a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 501a778
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16783/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16783/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16783/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 12 new or modified test files. 0 mvndep 1m 34s Maven dependency ordering for branch +1 mvninstall 7m 22s trunk passed +1 compile 6m 46s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 3m 21s trunk passed +1 mvneclipse 1m 14s trunk passed +1 findbugs 5m 28s trunk passed +1 javadoc 2m 38s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 3m 29s the patch passed +1 compile 8m 22s the patch passed +1 javac 8m 22s the patch passed +1 checkstyle 1m 39s root: The patch generated 0 new + 808 unchanged - 6 fixed = 808 total (was 814) +1 mvnsite 3m 54s the patch passed +1 mvneclipse 1m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 7m 25s the patch passed +1 javadoc 2m 57s the patch passed +1 unit 8m 31s hadoop-common in the patch passed. +1 unit 2m 11s hadoop-kms in the patch passed. +1 unit 1m 3s hadoop-hdfs-client in the patch passed. -1 unit 64m 37s hadoop-hdfs in the patch failed. +1 unit 3m 40s hadoop-hdfs-httpfs in the patch passed. +1 unit 1m 50s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 143m 17s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10489 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828904/HDFS-10489.06.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 24672b0bd19a 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 501a778 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16783/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16783/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16783/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Committed to trunk, thanks for the great reviews Andrew!

        Show
        xiaochen Xiao Chen added a comment - Committed to trunk, thanks for the great reviews Andrew!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10457 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10457/)
        HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS (xiao: rev ea839bd48e4478fc7b6d0a69e0eaeae2de5e0f0d)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
        • (edit) hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm
        • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestRpcProgramNfs3.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
        • (edit) hadoop-common-project/hadoop-common/src/site/markdown/DeprecatedProperties.md
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReservedRawPaths.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSecureEncryptionZoneWithKMS.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md
        • (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/TestCryptoAdminCLI.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestKeyProviderCache.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/KeyProviderCache.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithHA.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderFactory.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
        • (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestAclsEndToEnd.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10457 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10457/ ) HDFS-10489 . Deprecate dfs.encryption.key.provider.uri for HDFS (xiao: rev ea839bd48e4478fc7b6d0a69e0eaeae2de5e0f0d) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java (edit) hadoop-common-project/hadoop-kms/src/site/markdown/index.md.vm (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestRpcProgramNfs3.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java (edit) hadoop-common-project/hadoop-common/src/site/markdown/DeprecatedProperties.md (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReservedRawPaths.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSecureEncryptionZoneWithKMS.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/TestCryptoAdminCLI.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestKeyProviderCache.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/KeyProviderCache.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZonesWithHA.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderFactory.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestAclsEndToEnd.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
        Hide
        andrew.wang Andrew Wang added a comment -

        Hey Xiao, do you want to commit this to branch-2 and branch-2.8 also? We're allowed to deprecate keys during a minor release, just not remove them.

        Show
        andrew.wang Andrew Wang added a comment - Hey Xiao, do you want to commit this to branch-2 and branch-2.8 also? We're allowed to deprecate keys during a minor release, just not remove them.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Andrew Wang for the suggestion.
        I removed the 'Incompatible Change' flag accordingly, will push to branch-2/2.8 momentarily.

        Show
        xiaochen Xiao Chen added a comment - Thanks Andrew Wang for the suggestion. I removed the 'Incompatible Change' flag accordingly, will push to branch-2/2.8 momentarily.
        Hide
        xiaochen Xiao Chen added a comment - - edited

        Committed this to branch-2 and branch-2.8. Compiled locally before pushing.

        Show
        xiaochen Xiao Chen added a comment - - edited Committed this to branch-2 and branch-2.8. Compiled locally before pushing.

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development