Details

    • Type: New Feature New Feature
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: security
    • Labels:
    • Release Note:
      Initial patch for CMF functionality
    • Target Version/s:

      Description

      The credential management framework consists of library for securing, acquiring and rolling credentials for a given Hadoop service.

      Specifically the library will provide:

      1. Password Indirection or Aliasing
      2. Management of identity and trust keystores
      3. Rolling of key pairs and credentials
      4. Discovery of externally provisioned credentials
      5. Service specific CMF secret protection
      6. Syntax for Aliases within configuration files

      Password Indirection or Aliasing:
      By providing alias based access to actual secrets stored within a service specific JCEKS keystore, we are able to eliminate the need for any secret to be stored in clear text on the filesystem. This is a current redflag during security reviews for many customers.

      Management of Identity and Trust Keystores:
      Service specific identity and trust keystores will be managed by a combination of the HSSO service and CMF.

      Upon registration with the HSSO service a dependent service will be able discover externally provisioned keystores or have them created by the HSSO service on its behalf. The public key of the HSSO service will be provided to the service to be imported into its service specific trust store.

      Service specific keystores and credential stores will be protected with the service specific CMF secret.

      Rolling of Keypairs and Credentials:
      The ability to automate the rolling of PKI keypairs and credentials provide the services a common facility for discovering new HSSO public keys and the need and means to roll their own credentials while being able to retain a number of previous values (as needed).

      Discovery of Externally Provisioned Credentials:
      For environments that want control over the certificate generation and provisioning, CMF provides the ability to discover preprovisioned artifacts based on naming conventions of the artifacts and the use of the service specific CMF secret to access the credentials within the keystores.

      Service Specific CMF Secret Protection:
      By providing a common facility to prompt for and optionally persist a service specific CMF secret at service installation/startup, we enable the ability to protect all the service specific security artifacts with this protected secret. It is protected with a combination of AES 128 bit encryption and file permissions set for only the service specific OS user.

      Syntax for Aliases within configuration files:
      In order to facilitate the use of aliases but also preserve backward compatibility of config files, we will introduce a syntax for marking a value in a configuration file as an alias. A getSecret(String value) type utility method will encapsulate the recognition and parsing of an alias and the retrieval from CMF or return the provided value as the password.

      For instance, if a properties file were to require a password to be provided instead of:

      passwd=supersecret

      we would provide an alias as such:

      passwd=$

      {ALIAS=supersecret}

      At runtime, the value from the properties file is provided to the CMF.getSecret(value) method and it either resolves the alias (where it finds the alias syntax) or returns the value (when there is no alias syntax).

      1. HADOOP-9534.patch
        88 kB
        Larry McCay
      2. CMF-overview.txt
        6 kB
        Larry McCay
      3. HADOOP-9534.patch
        88 kB
        Larry McCay
      4. HADOOP-9534.patch
        88 kB
        Larry McCay
      5. HADOOP-9534.patch
        88 kB
        Larry McCay
      6. 0002-HADOOP-9534-Credential-Management-Framework-second-iteration-.patch
        82 kB
        Larry McCay
      7. 0001-HADOOP-9534-Credential-Management-Framework-initial-.patch
        83 kB
        Larry McCay

        Issue Links

          Activity

          Owen O'Malley made changes -
          Link This issue relates to HDFS-5143 [ HDFS-5143 ]
          Larry McCay made changes -
          Link This issue is related to HIVE-4227 [ HIVE-4227 ]
          Larry McCay made changes -
          Parent HADOOP-9533 [ 12645641 ]
          Issue Type Sub-task [ 7 ] New Feature [ 2 ]
          Larry McCay made changes -
          Link This issue is related to HADOOP-9825 [ HADOOP-9825 ]
          Larry McCay made changes -
          Attachment HADOOP-9534.patch [ 12595769 ]
          Larry McCay made changes -
          Attachment CMF-overview.txt [ 12595768 ]
          Larry McCay made changes -
          Link This issue is related to HADOOP-9811 [ HADOOP-9811 ]
          Larry McCay made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Larry McCay made changes -
          Attachment HADOOP-9534.patch [ 12595177 ]
          Larry McCay made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jitendra Nath Pandey made changes -
          Assignee Larry McCay [ lmccay ]
          Larry McCay made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Larry McCay made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Larry McCay made changes -
          Attachment HADOOP-9534.patch [ 12595034 ]
          Larry McCay made changes -
          Link This issue relates to HADOOP-9392 [ HADOOP-9392 ]
          Larry McCay made changes -
          Link This issue is required by HADOOP-9781 [ HADOOP-9781 ]
          Larry McCay made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Larry McCay made changes -
          Attachment HADOOP-9534.patch [ 12594717 ]
          Larry McCay made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Larry McCay made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Larry McCay made changes -
          Larry McCay made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Larry McCay made changes -
          Larry McCay made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Initial patch for CMF functionality
          Affects Version/s 3.0.0 [ 12320357 ]
          Target Version/s 3.0.0 [ 12320357 ]
          Labels patch
          Larry McCay made changes -
          Field Original Value New Value
          Description The credential management framework consists of library for securing, acquiring and rolling credentials for a given Hadoop service.

          Specifically the library will provide:

          1. Password Indirection or Aliasing
          2. Management of identity and trust keystores
          3. Rolling of key pairs and credentials
          4. Discovery of externally provisioned credentials
          5. Service specific CMF secret protection

          Password Indirection or Aliasing:
          By providing alias based access to actual secrets stored within a service specific JCEKS keystore, we are able to eliminate the need for any secret to be stored in clear text on the filesystem. This is a current redflag during security reviews for many customers.

          Management of Identity and Trust Keystores:
          Service specific identity and trust keystores will be managed by a combination of the HSSO service and CMF.

          Upon registration with the HSSO service a dependent service will be able discover externally provisioned keystores or have them created by the HSSO service on its behalf. The public key of the HSSO service will be provided to the service to be imported into its service specific trust store.

          Service specific keystores and credential stores will be protected with the service specific CMF secret.

          Rolling of Keypairs and Credentials:
          The ability to automate the rolling of PKI keypairs and credentials provide the services a common facility for discovering new HSSO public keys and the need and means to roll their own credentials while being able to retain a number of previous values (as needed).

          Discovery of Externally Provisioned Credentials:
          For environments that want control over the certificate generation and provisioning, CMF provides the ability to discover preprovisioned artifacts based on naming conventions of the artifacts and the use of the service specific CMF secret to access the credentials within the keystores.

          Service Specific CMF Secret Protection:
          By providing a common facility to prompt for and optionally persist a service specific CMF secret at service installation/startup, we enable the ability to protect all the service specific security artifacts with this protected secret. It is protected with a combination of AES 128 bit encryption and file permissions set for only the service specific OS user.
          The credential management framework consists of library for securing, acquiring and rolling credentials for a given Hadoop service.

          Specifically the library will provide:

          1. Password Indirection or Aliasing
          2. Management of identity and trust keystores
          3. Rolling of key pairs and credentials
          4. Discovery of externally provisioned credentials
          5. Service specific CMF secret protection
          6. Syntax for Aliases within configuration files

          Password Indirection or Aliasing:
          By providing alias based access to actual secrets stored within a service specific JCEKS keystore, we are able to eliminate the need for any secret to be stored in clear text on the filesystem. This is a current redflag during security reviews for many customers.

          Management of Identity and Trust Keystores:
          Service specific identity and trust keystores will be managed by a combination of the HSSO service and CMF.

          Upon registration with the HSSO service a dependent service will be able discover externally provisioned keystores or have them created by the HSSO service on its behalf. The public key of the HSSO service will be provided to the service to be imported into its service specific trust store.

          Service specific keystores and credential stores will be protected with the service specific CMF secret.

          Rolling of Keypairs and Credentials:
          The ability to automate the rolling of PKI keypairs and credentials provide the services a common facility for discovering new HSSO public keys and the need and means to roll their own credentials while being able to retain a number of previous values (as needed).

          Discovery of Externally Provisioned Credentials:
          For environments that want control over the certificate generation and provisioning, CMF provides the ability to discover preprovisioned artifacts based on naming conventions of the artifacts and the use of the service specific CMF secret to access the credentials within the keystores.

          Service Specific CMF Secret Protection:
          By providing a common facility to prompt for and optionally persist a service specific CMF secret at service installation/startup, we enable the ability to protect all the service specific security artifacts with this protected secret. It is protected with a combination of AES 128 bit encryption and file permissions set for only the service specific OS user.

          Syntax for Aliases within configuration files:
          In order to facilitate the use of aliases but also preserve backward compatibility of config files, we will introduce a syntax for marking a value in a configuration file as an alias. A getSecret(String value) type utility method will encapsulate the recognition and parsing of an alias and the retrieval from CMF or return the provided value as the password.

          For instance, if a properties file were to require a password to be provided instead of:

          passwd=supersecret

          we would provide an alias as such:

          passwd=${ALIAS=supersecret}

          At runtime, the value from the properties file is provided to the CMF.getSecret(value) method and it either resolves the alias (where it finds the alias syntax) or returns the value (when there is no alias syntax).
          Larry McCay created issue -

            People

            • Assignee:
              Larry McCay
              Reporter:
              Larry McCay
            • Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 504h
                504h
                Remaining:
                Remaining Estimate - 504h
                504h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development