Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
This task tracks the required changes in the handling code for the Blueprint .json and the Cluster Creation Template .json files in order to allow the user to request that a given cluster be Kerberized.
The most natural place for this configuration will likely be in the Cluster Creation template, which would then allow a given Blueprint to be referenced via secure and non-secure cluster creation requests.
Based on feedback from Product Management, a customer should be able to indicate that a cluster is to be Kerberized in either the Blueprint .json or the Cluster Creation template .json.
This feature should support enabling Kerberos at the level of the Blueprint or the Cluster Creation template. In either JSON document, the user should be able to indicate a security tag that looks like:
"security" : { "type" : "KERBEROS", "kerberos_descriptor_reference" : "kd1", "kerberos_descriptor" : { ... } }
The "type" field in the new "security" map should be set to "KERBEROS" in order to indicate that Kerberos should be supported.
The "kerberos_descriptor_reference" field in the "security" map could be used to refer to an existing Kerberos descriptor that has been POST-ed to the Ambari REST API.
If the user wishes to embed the Kerberos descriptor in the Blueprint or Cluster Creation template, then the "kerberos_descriptor" field in the "security" map should be set to the contents of that descriptor.
The "security" map could eventually also include other configuration items pertaining to the security of a given cluster. While Kerberos is the initial support being added, other security mechanisms may evolve over time, and we should be able to use the same configuration structures in order to eventually integrate with these technologies as well.
Note: The user should typically only specify a "kerberos_descriptor_reference" or a "kerberos_descriptor". If both are set, the Blueprint processor should treat this as an error condition.
This new JSON element should exist at the top-level of the Cluster Creation Template and Blueprint documents.
The following example shows what a Cluster Creation template might look like in this scenario:
{
"blueprint" : "blueprint-ha",
"default_password" : "default",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
},
"host_groups" :[
{
"name" : "host_group_1",
"hosts" : [
{
"fqdn" : "c6401.ambari.apache.org"
}
]
},
...
]
}
The following example shows what a Blueprint that requires Kerberos support should look like:
{
"host_groups": [
{
"name": "master",
"configurations": [
...
],
"components": [
{
"name": "NAMENODE"
},
{
"name": "SECONDARY_NAMENODE"
},
{
"name": "RESOURCEMANAGER"
},
{
"name": "HISTORYSERVER"
},
{
"name": "APP_TIMELINE_SERVER"
},
{
"name": "ZOOKEEPER_SERVER"
}
],
"cardinality": "1"
},
...
],
"Blueprints": {
"blueprint_name": "multi-simple-yarn",
"stack_name": "HDP",
"stack_version": "2.2",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
}
}
}
In the example above, the "type" field is included in the "security" map section of the Blueprint document, embedded within the "Blueprints" map. This is the most natural place for the Blueprint itself, since it contains the metadata that should be associated with the Blueprint deployment, outside of the configuration and components.
Priority Ordering
Since the Kerberos setting will be supported in either the Blueprint or the Cluster Creation template, this new support will need to handle the cases where the setting is chosen in both documents.
- If a security type of "KERBEROS" is not selected in a Blueprint, then the Cluster Creation template used by override this setting by including "type" : "KERBEROS" in the template. This allows us to support deploying a Blueprint in both Kerberized and non-Kerberized environments. This implies that any Kerberos-specific configuration would need to be included in the Cluster Creation template, but this is already supported by the Blueprints configuration overrides.
- If a security type of "KERBEROS" is selected, then the Cluster Creation template should not be able to override this setting to less-secure mode. If the Cluster Creation template is configured to use a different security mechanism, (For example: "type" : "OFF"), then the Blueprints processor should treat this as an error condition. If the Cluster creation template does not specify a "security" tag, then the "security" setting in the Blueprint should be honored. In general, we should allow overrides to a more-secure cluster, and forbid overrides for a less-secure cluster.
Blueprint Database Table Changes
These additions to the Blueprint .json and Cluster Creation Template .json structure will likely require changes to the Blueprint entity database tables, already defined in ambari-server.
This current task will encompass any Database table changes needed to make these additions, and will also likely require some ambari-server Upgrade handling. This will involve using the existing Ambari Upgrade utilities to support moving from older Ambari installs to Ambari 2.2. The main work here will be updating existing database tables to support the new structure.
Backwards compatibility
Any Blueprints that worked in previous versions of Ambari (non-Kerberized) should work as-is in Ambari 2.2, in order to preserve backwards compatibility. This means that these new configuration tags must not be required in a non-Kerberized environment.
Blueprint Validation
The Blueprint validator should be updated to check on the value of the security "type" field, when it is present. Once we determine the accepted set of possible values ("OFF" and "KERBEROS", for now), the validator should check this, and return a reasonable error to the REST client if an invalid value is set.
The kerberos.json (either referenced or embedded) descriptor must be saved to the cluster’s artifacts resource prior to Kerberization.
In order to setup Kerberos, KDC admin credentials must be available, users can choose the store type when submitting KDC admin credentials inside Cluster template:
"credentials" : [ { "alias" : "kdc.admin.credential", "principal" : "(PRINCIPAL)", "key" : "(PASSWORD)", "type" : "(temporary | persistent)" } ]
They can either set type=PERSISTENT or TEMPORARY. Temporary admin credentials are valid 90 minutes or until server restart. Should they want to add new hosts after temporary credentials are expired they had to submit credentials using credential API:
POST /api/v1/clusters/:clusterName/credentials/kdc.admin.credential
{
"Credential" : {
"principal" : "(PRINCIPAL)",
"key" : "(PASSWORD)",
"type" : "(persisted|temporary)"
}
}