Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9534

Credential Management Framework (CMF)



    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.0.0-alpha1
    • None
    • security
    • Initial patch for CMF functionality


      The credential management framework consists of library for securing, acquiring and rolling credentials for a given Hadoop service.

      Specifically the library will provide:

      1. Password Indirection or Aliasing
      2. Management of identity and trust keystores
      3. Rolling of key pairs and credentials
      4. Discovery of externally provisioned credentials
      5. Service specific CMF secret protection
      6. Syntax for Aliases within configuration files

      Password Indirection or Aliasing:
      By providing alias based access to actual secrets stored within a service specific JCEKS keystore, we are able to eliminate the need for any secret to be stored in clear text on the filesystem. This is a current redflag during security reviews for many customers.

      Management of Identity and Trust Keystores:
      Service specific identity and trust keystores will be managed by a combination of the HSSO service and CMF.

      Upon registration with the HSSO service a dependent service will be able discover externally provisioned keystores or have them created by the HSSO service on its behalf. The public key of the HSSO service will be provided to the service to be imported into its service specific trust store.

      Service specific keystores and credential stores will be protected with the service specific CMF secret.

      Rolling of Keypairs and Credentials:
      The ability to automate the rolling of PKI keypairs and credentials provide the services a common facility for discovering new HSSO public keys and the need and means to roll their own credentials while being able to retain a number of previous values (as needed).

      Discovery of Externally Provisioned Credentials:
      For environments that want control over the certificate generation and provisioning, CMF provides the ability to discover preprovisioned artifacts based on naming conventions of the artifacts and the use of the service specific CMF secret to access the credentials within the keystores.

      Service Specific CMF Secret Protection:
      By providing a common facility to prompt for and optionally persist a service specific CMF secret at service installation/startup, we enable the ability to protect all the service specific security artifacts with this protected secret. It is protected with a combination of AES 128 bit encryption and file permissions set for only the service specific OS user.

      Syntax for Aliases within configuration files:
      In order to facilitate the use of aliases but also preserve backward compatibility of config files, we will introduce a syntax for marking a value in a configuration file as an alias. A getSecret(String value) type utility method will encapsulate the recognition and parsing of an alias and the retrieval from CMF or return the provided value as the password.

      For instance, if a properties file were to require a password to be provided instead of:


      we would provide an alias as such:


      At runtime, the value from the properties file is provided to the CMF.getSecret(value) method and it either resolves the alias (where it finds the alias syntax) or returns the value (when there is no alias syntax).


        1. HADOOP-9534.patch
          88 kB
          Larry McCay
        2. HADOOP-9534.patch
          88 kB
          Larry McCay
        3. HADOOP-9534.patch
          88 kB
          Larry McCay
        4. HADOOP-9534.patch
          88 kB
          Larry McCay
        5. CMF-overview.txt
          6 kB
          Larry McCay
        6. 0002-HADOOP-9534-Credential-Management-Framework-second-iteration-.patch
          82 kB
          Larry McCay
        7. 0001-HADOOP-9534-Credential-Management-Framework-initial-.patch
          83 kB
          Larry McCay

        Issue Links



              lmccay Larry McCay
              lmccay Larry McCay
              0 Vote for this issue
              24 Start watching this issue



                Time Tracking

                  Original Estimate - 504h
                  Remaining Estimate - 504h
                  Time Spent - Not Specified
                  Not Specified