Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10433

Key Management Server based on KeyProvider API

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha1
    • 2.6.0
    • security
    • None
    • Reviewed


      (from HDFS-6134 proposal)

      Hadoop KMS is the gateway, for Hadoop and Hadoop clients, to the underlying KMS. It provides an interface that works with existing Hadoop security components (authenticatication, confidentiality).

      Hadoop KMS will be implemented leveraging the work being done in HADOOP-10141 and HADOOP-10177.

      Hadoop KMS will provide an additional implementation of the Hadoop KeyProvider class. This implementation will be a client-server implementation.

      The client-server protocol will be secure:

      • Kerberos HTTP SPNEGO (authentication)
      • HTTPS for transport (confidentiality and integrity)
      • Hadoop ACLs (authorization)

      The Hadoop KMS implementation will not provide additional ACL to access encrypted files. For sophisticated access control requirements, HDFS ACLs (HDFS-4685) should be used.

      Basic key administration will be supported by the Hadoop KMS via the, already available, Hadoop KeyShell command line tool

      There are minor changes that must be done in Hadoop KeyProvider functionality:

      The KeyProvider contract, and the existing implementations, must be thread-safe

      KeyProvider API should have an API to generate the key material internally
      JavaKeyStoreProvider should use, if present, a password provided via configuration

      KeyProvider Option and Metadata should include a label (for easier cross-referencing)

      To avoid overloading the underlying KeyProvider implementation, the Hadoop KMS will cache keys using a TTL policy.

      Scalability and High Availability of the Hadoop KMS can achieved by running multiple instances behind a VIP/Load-Balancer. For High Availability, the underlying KeyProvider implementation used by the Hadoop KMS must be High Available.


        1. KMS-doc.pdf
          237 kB
          Alejandro Abdelnur
        2. HadoopKMSDocsv2.pdf
          527 kB
          Alejandro Abdelnur
        3. HADOOP-10433.patch
          183 kB
          Alejandro Abdelnur
        4. HADOOP-10433.patch
          183 kB
          Alejandro Abdelnur
        5. HADOOP-10433.patch
          201 kB
          Alejandro Abdelnur
        6. HADOOP-10433.patch
          201 kB
          Alejandro Abdelnur
        7. HADOOP-10433.patch
          201 kB
          Alejandro Abdelnur
        8. HADOOP-10433.patch
          202 kB
          Alejandro Abdelnur
        9. HADOOP-10433.patch
          203 kB
          Alejandro Abdelnur
        10. HADOOP-10433.patch
          203 kB
          Alejandro Abdelnur
        11. HADOOP-10433.patch
          191 kB
          Alejandro Abdelnur
        12. HADOOP-10433.patch
          191 kB
          Alejandro Abdelnur
        13. HADOOP-10433.patch
          191 kB
          Alejandro Abdelnur
        14. HADOOP-10433.patch
          192 kB
          Alejandro Abdelnur

        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            tucu00 Alejandro Abdelnur Assign to me
            tucu00 Alejandro Abdelnur
            0 Vote for this issue
            20 Start watching this issue




                Issue deployment