Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13443

KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: kms
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      By default, the KMS wraps the active key provider in a CachingKeyProvider at runtime. This prevents the KeyProviderCryptoExtension.createKeyProviderCryptoExtension method from ever detecting whether the active key provider implements theKeyProviderCryptoExtension interface. Therefore, the DefaultCryptoExtension is always used.

      1. HADOOP-13443.patch
        8 kB
        Anthony Young-Garner
      2. HADOOP-13443.patch
        11 kB
        Anthony Young-Garner
      3. HADOOP-13443.03.patch
        11 kB
        Anthony Young-Garner

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 26s trunk passed
        +1 compile 7m 39s trunk passed
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 1m 3s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 27s trunk passed
        +1 javadoc 0m 46s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 7m 59s the patch passed
        +1 javac 7m 59s the patch passed
        -0 checkstyle 0m 23s hadoop-common-project/hadoop-common: The patch generated 10 new + 13 unchanged - 0 fixed = 23 total (was 13)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 27s the patch passed
        +1 javadoc 0m 44s the patch passed
        +1 unit 7m 8s hadoop-common in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        41m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821095/HADOOP-13443.patch
        JIRA Issue HADOOP-13443
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 39955467d57e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 95f2b98
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 26s trunk passed +1 compile 7m 39s trunk passed +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 27s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 7m 59s the patch passed +1 javac 7m 59s the patch passed -0 checkstyle 0m 23s hadoop-common-project/hadoop-common: The patch generated 10 new + 13 unchanged - 0 fixed = 23 total (was 13) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 27s the patch passed +1 javadoc 0m 44s the patch passed +1 unit 7m 8s hadoop-common in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 41m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821095/HADOOP-13443.patch JIRA Issue HADOOP-13443 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 39955467d57e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 95f2b98 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10126/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks very much for finding and fixing this, Anthony Young-Garner!

        Patch looks good in general. A few comments:

        • Before this patch, if the KeyProvider itself implements CryptoExtension, it will be used, no matter whether it implements KeyProviderExtension or not. This behavior is changed from this patch, is it what we intend to do? IMHO we should check on CryptoExtension first, and add the KeyProviderExtension check as a fall back of the former.
        • In patch 1, if keyProvider instanceof KeyProviderExtension == true but keyProviderExtension.getKeyProvider() instanceof KeyProviderCryptoExtension.CryptoExtension == false, cryptoExtension will end up being null. Let's make sure the default is used in any case.
        • Please fix the checkstyle warnings.
        Show
        xiaochen Xiao Chen added a comment - Thanks very much for finding and fixing this, Anthony Young-Garner ! Patch looks good in general. A few comments: Before this patch, if the KeyProvider itself implements CryptoExtension , it will be used, no matter whether it implements KeyProviderExtension or not. This behavior is changed from this patch, is it what we intend to do? IMHO we should check on CryptoExtension first, and add the KeyProviderExtension check as a fall back of the former. In patch 1, if keyProvider instanceof KeyProviderExtension == true but keyProviderExtension.getKeyProvider() instanceof KeyProviderCryptoExtension.CryptoExtension == false , cryptoExtension will end up being null . Let's make sure the default is used in any case. Please fix the checkstyle warnings.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 31s trunk passed
        +1 compile 6m 54s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 17s trunk passed
        +1 javadoc 0m 45s trunk passed
        +1 mvninstall 0m 38s the patch passed
        +1 compile 6m 58s the patch passed
        +1 javac 6m 58s the patch passed
        -0 checkstyle 0m 31s hadoop-common-project/hadoop-common: The patch generated 10 new + 13 unchanged - 0 fixed = 23 total (was 13)
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 30s the patch passed
        +1 javadoc 0m 46s the patch passed
        -1 unit 17m 1s hadoop-common in the patch failed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        47m 30s



        Reason Tests
        Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821714/HADOOP-13443.patch
        JIRA Issue HADOOP-13443
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8fe5138b0fc6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d28c2d9
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 31s trunk passed +1 compile 6m 54s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 17s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 6m 58s the patch passed +1 javac 6m 58s the patch passed -0 checkstyle 0m 31s hadoop-common-project/hadoop-common: The patch generated 10 new + 13 unchanged - 0 fixed = 23 total (was 13) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 30s the patch passed +1 javadoc 0m 46s the patch passed -1 unit 17m 1s hadoop-common in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 47m 30s Reason Tests Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821714/HADOOP-13443.patch JIRA Issue HADOOP-13443 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8fe5138b0fc6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d28c2d9 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10158/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 59s trunk passed
        +1 compile 6m 45s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 19s trunk passed
        +1 javadoc 0m 44s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 6m 39s the patch passed
        +1 javac 6m 39s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 23s the patch passed
        +1 javadoc 0m 44s the patch passed
        +1 unit 8m 1s hadoop-common in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        38m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821729/HADOOP-13443.03.patch
        JIRA Issue HADOOP-13443
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c2ee141aa78c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d28c2d9
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10160/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10160/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 59s trunk passed +1 compile 6m 45s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 19s trunk passed +1 javadoc 0m 44s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 6m 39s the patch passed +1 javac 6m 39s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 23s the patch passed +1 javadoc 0m 44s the patch passed +1 unit 8m 1s hadoop-common in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 38m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821729/HADOOP-13443.03.patch JIRA Issue HADOOP-13443 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c2ee141aa78c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d28c2d9 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10160/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10160/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for revving Anthony, looks pretty good to me!
        +1, will commit soon.

        Show
        xiaochen Xiao Chen added a comment - Thanks for revving Anthony, looks pretty good to me! +1, will commit soon.
        Hide
        anthony.young-garner@cloudera.com Anthony Young-Garner added a comment -

        Thanks Xiao. Sorry for the churn. Uploaded too quickly the first time.

        Show
        anthony.young-garner@cloudera.com Anthony Young-Garner added a comment - Thanks Xiao. Sorry for the churn. Uploaded too quickly the first time.
        Hide
        xiaochen Xiao Chen added a comment -

        Committed to trunk and branch-2.
        Thanks very much Anthony for reporting and fixing the issue!

        Show
        xiaochen Xiao Chen added a comment - Committed to trunk and branch-2. Thanks very much Anthony for reporting and fixing the issue!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10211 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10211/)
        HADOOP-13443. KMS should check the type of underlying keyprovider of (xiao: rev 05db64913d8c620cf481f74219bac9b72cb142af)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderCryptoExtension.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyProviderCryptoExtension.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10211 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10211/ ) HADOOP-13443 . KMS should check the type of underlying keyprovider of (xiao: rev 05db64913d8c620cf481f74219bac9b72cb142af) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderCryptoExtension.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyProviderCryptoExtension.java

          People

          • Assignee:
            anthony.young-garner@cloudera.com Anthony Young-Garner
            Reporter:
            anthony.young-garner@cloudera.com Anthony Young-Garner
          • Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development