Hadoop Common
  1. Hadoop Common
  2. HADOOP-10750

KMSKeyProviderCache should be in hadoop-common

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 2.6.0
    • Component/s: security
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      KMS has KMSCacheKeyProvider, this class should be available in hadoop-common for users of KeyProvider instances to wrap them and avoid several, potentially expensive, key retrievals.

      1. HADOOP-10750.3.patch
        31 kB
        Arun Suresh
      2. HADOOP-10750.2.patch
        29 kB
        Arun Suresh
      3. HADOOP-10750.1.patch
        27 kB
        Arun Suresh

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1836 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1836/)
          HADOOP-10750. KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1836 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1836/ ) HADOOP-10750 . KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1809 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1809/)
          HADOOP-10750. KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1809 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1809/ ) HADOOP-10750 . KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #617 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/617/)
          HADOOP-10750. KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #617 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/617/ ) HADOOP-10750 . KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #5919 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5919/)
          HADOOP-10750. KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm
          • /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #5919 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5919/ ) HADOOP-10750 . KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611823 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/CachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestCachingKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSCacheKeyProvider.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSConfiguration.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/site/apt/index.apt.vm /hadoop/common/trunk/hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSCacheKeyProvider.java
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Arun. Committed to trunk.

          Show
          Alejandro Abdelnur added a comment - Thanks Arun. Committed to trunk.
          Hide
          Alejandro Abdelnur added a comment -

          +1.

          Show
          Alejandro Abdelnur added a comment - +1.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12656562/HADOOP-10750.3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms:

          org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem
          org.apache.hadoop.ipc.TestIPC
          org.apache.hadoop.fs.TestSymlinkLocalFSFileContext

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4316//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4316//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12656562/HADOOP-10750.3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms: org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem org.apache.hadoop.ipc.TestIPC org.apache.hadoop.fs.TestSymlinkLocalFSFileContext +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4316//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4316//console This message is automatically generated.
          Hide
          Arun Suresh added a comment -

          Updating patch addressing feedback. Thanks Alejandro Abdelnur

          Show
          Arun Suresh added a comment - Updating patch addressing feedback. Thanks Alejandro Abdelnur
          Hide
          Alejandro Abdelnur added a comment -

          A few minor things and we are ready to go:

          CachingKeyProvider.java:

          • Remove serialVersionUID from KeyNotFoundException
          • no need for getCacheExtension(), getExtension() is generified
          • the 2 rollNewVersion() methods must invalidate the metadatacache

          KMSConfiguration.java:
          . KEY_CACHE_TIMEOUT_DEFAULT should be higher, i.e. 10 mins

          KMSWebApp.java:
          . we should have a config property to disable cache, not wrapping the keyprovider at all. default value, caching == TRUE

          KMS doc:
          . it should mention caching configs, timeouts and enable/disable

          Show
          Alejandro Abdelnur added a comment - A few minor things and we are ready to go: CachingKeyProvider.java : Remove serialVersionUID from KeyNotFoundException no need for getCacheExtension() , getExtension() is generified the 2 rollNewVersion() methods must invalidate the metadatacache KMSConfiguration.java : . KEY_CACHE_TIMEOUT_DEFAULT should be higher, i.e. 10 mins KMSWebApp.java : . we should have a config property to disable cache, not wrapping the keyprovider at all. default value, caching == TRUE KMS doc : . it should mention caching configs, timeouts and enable/disable
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12655843/HADOOP-10750.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms:

          org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
          org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
          org.apache.hadoop.ipc.TestIPC
          org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4279//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4279//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655843/HADOOP-10750.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms: org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl org.apache.hadoop.fs.TestSymlinkLocalFSFileContext org.apache.hadoop.ipc.TestIPC org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4279//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4279//console This message is automatically generated.
          Hide
          Arun Suresh added a comment -

          Alejandro Abdelnur, yup.. I chose to extend KeyProviderExtension since it already delegates all methods to a provided KeyProvider class

          Updated Patch : Made the "currentKeyTimeout" configurable.
          Earlier the Key timeout itself was 10 sec.. The "currentKeytimeout" should ideally be less than that (which was why it was 1 sec). I have thus updated both the keyTimeout and currentKeyTimeout default values to 60 and 30 secs respectively.

          Show
          Arun Suresh added a comment - Alejandro Abdelnur , yup.. I chose to extend KeyProviderExtension since it already delegates all methods to a provided KeyProvider class Updated Patch : Made the "currentKeyTimeout" configurable. Earlier the Key timeout itself was 10 sec.. The "currentKeytimeout" should ideally be less than that (which was why it was 1 sec). I have thus updated both the keyTimeout and currentKeyTimeout default values to 60 and 30 secs respectively.
          Hide
          Alejandro Abdelnur added a comment -

          Does the CachingKeyProvider need to be an extension? Or is just simply not to override all provider methods?

          The timeout for currentKeys should be configurable with a default of 30 or 60 secs (1 sec is too aggressive)

          Show
          Alejandro Abdelnur added a comment - Does the CachingKeyProvider need to be an extension? Or is just simply not to override all provider methods? The timeout for currentKeys should be configurable with a default of 30 or 60 secs (1 sec is too aggressive)
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12654389/HADOOP-10750.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4222//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4222//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12654389/HADOOP-10750.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4222//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4222//console This message is automatically generated.
          Hide
          Arun Suresh added a comment -

          Uploaded Patch :

          • Moved KMSCacheKeyProvider to hadoop-common CachingKeyProvider. This class implements the KeyProviderExtension abstract class which, by default, delegates all methods to the underlying KeyProvider and over-rides the get KeyVersion/CurrentVersion/Metadata methods to delegate to a inner static CacheExtension
          Show
          Arun Suresh added a comment - Uploaded Patch : Moved KMSCacheKeyProvider to hadoop-common CachingKeyProvider . This class implements the KeyProviderExtension abstract class which, by default, delegates all methods to the underlying KeyProvider and over-rides the get KeyVersion/CurrentVersion/Metadata methods to delegate to a inner static CacheExtension
          Hide
          Alejandro Abdelnur added a comment -

          BTW, the cache should also cache key metadata entries.

          Show
          Alejandro Abdelnur added a comment - BTW, the cache should also cache key metadata entries.
          Hide
          Alejandro Abdelnur added a comment - - edited

          On #1, correct. Plus both expiration times should be configurable.

          On #2, KMS is an obvious candidate. KeyProvider implementations that want to manage their own caching as well. And any service using KeyProviders, i.e. Oozie, NN, httpfs, Knox, etc. Even AMs of Yarn Jobs could benefit from it.

          Show
          Alejandro Abdelnur added a comment - - edited On #1, correct. Plus both expiration times should be configurable. On #2, KMS is an obvious candidate. KeyProvider implementations that want to manage their own caching as well. And any service using KeyProviders, i.e. Oozie, NN, httpfs, Knox, etc. Even AMs of Yarn Jobs could benefit from it.
          Hide
          Larry McCay added a comment -

          Interesting...

          I have a couple high-level comments/questions:

          1. if this is a generic cache provider then it would make sense to remove the KMS part of the name and any dependencies within - if there are any.
          2. I'm not sure that I can envision the typical usage pattern for it. In order for the cache to be meaningful it would have to be used across many access attempts. This means that it would need to be an instance of the cache provider (or a backend store of some sort) that lives across multiple usages. I can easily see how it can be used in a server type environment such as KMS but for general use it isn't quite as clear. Can you describe the general use that you have in mind?

          Show
          Larry McCay added a comment - Interesting... I have a couple high-level comments/questions: 1. if this is a generic cache provider then it would make sense to remove the KMS part of the name and any dependencies within - if there are any. 2. I'm not sure that I can envision the typical usage pattern for it. In order for the cache to be meaningful it would have to be used across many access attempts. This means that it would need to be an instance of the cache provider (or a backend store of some sort) that lives across multiple usages. I can easily see how it can be used in a server type environment such as KMS but for general use it isn't quite as clear. Can you describe the general use that you have in mind?

            People

            • Assignee:
              Arun Suresh
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development