Details
-
Epic
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.0.0
-
Kerberos
-
Ambari Automated Kerberization
Description
Problem
Manually installing and setting up Kerberos for a secure Hadoop cluster is error prone, largely manual and a potential source of configuration problems. It requires many steps where configuration files and credentials may need to be distributed across many nodes. Because of this the process is time consuming and lead to a high probability of user error.
The problem is exacerbated when the cluster is modified by adding or removing nodes and services.
Solution
Use Ambari to secure the cluster using Kerberos. By automating the process of setting up Kerberos, the repetitive tasks of distributing configuration details and credentials can be done in parallel to the nodes within the cluster. This also negates most user-related errors due to the lack of interaction a user has with the process.
See AmbariClusterKerberization.pdf for more details.
Attachments
Attachments
Issue Links
- is depended upon by
-
SLIDER-698 AM keytab creation and distribution in Ambari managed cluster
- Open
- requires
-
AMBARI-7985 Allow for server-side commands
- Resolved
-
AMBARI-8454 Create server-side actions to create kerberos principals and keytabs
- Resolved
-
AMBARI-7302 stacks API should provide information for features supported by a service or serviceComponent
- Open
Issues in epic
|
AMBARI-7448 | Create Kerberos Service | Resolved | Robert Levas | ||
|
AMBARI-7449 | Update API to enable configuring of services to use Kerberos | Resolved | Robert Levas | ||
|
AMBARI-7450 | Add UI Wizard to facilitate configuring cluster to use Kerberos | Resolved | Jaimin Jetly | ||
|
AMBARI-8163 | Provide stage resource information via REST API | Resolved | Tom Beerbower | ||
|
AMBARI-8166 | Implement custom command for checking connectivity to KDC, via REST API | Resolved | Rishi Pidva | ||
|
AMBARI-8247 | Provide a way to get service-specific Kerberos descriptor via REST API | Resolved | Robert Levas | ||
|
AMBARI-8277 | Create facility to get and set kerberos plans via REST API | Resolved | Robert Levas | ||
|
AMBARI-8336 | Add Security State to Ambari database | Resolved | Robert Levas | ||
|
AMBARI-8343 | Components should indicate Security State (via ambari-agent) | Resolved | Robert Levas | ||
|
AMBARI-8356 | Push kerberos keytabs from ambari server to appropriate service component host | Resolved | Dilli Arumugam | ||
|
AMBARI-8358 | update service configs for kerberos related configs when kerberos is enabled | Resolved | Dilli Arumugam | ||
|
AMBARI-8426 | Provide access to session from resource handler/provider | Resolved | Tom Beerbower | ||
|
AMBARI-8436 | Kerberos wizard: Test kerberos command fails when no credentials specified for test user | Resolved | Robert Levas | ||
|
AMBARI-8447 | Update ConfigurationResourceProvider to handle Kerberos Administrative Credentials as a special case | Resolved | Tom Beerbower | ||
|
AMBARI-8454 | Create server-side actions to create kerberos principals and keytabs | Resolved | Robert Levas | ||
|
AMBARI-8477 | HDFS service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8478 | Falcon service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8479 | YARN service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8480 | Kerberos service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8481 | Flume service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8482 | HBase service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8485 | Hive service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8486 | Kafka service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8487 | Oozie service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8488 | Pig service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8542 | Provide a way to parse and handle Kerberos descriptors | Resolved | Robert Levas | ||
|
AMBARI-8578 | Update Apache Directory Server Library from 1.5.5 to 2.0.0-M19 | Resolved | Robert Levas | ||
|
AMBARI-8627 | Allow for service-level Kerberos descriptor to contain multiple services | Resolved | Robert Levas | ||
|
AMBARI-8628 | Create Kerberos Descriptors for HDFS, YARN, MAPREDUCE2, HBASE, and HIVE services | Resolved | Robert Levas | ||
|
AMBARI-8647 | Pass Injector to ServerActionExecutor so objects can be injected into ServerAction implementations | Resolved | Robert Levas | ||
|
AMBARI-8657 | Add Kerberos Configuration Metadata File Builder and Reader | Resolved | Robert Levas | ||
|
AMBARI-8660 | KerberosCredential class should encrypt and decrypt itself | Resolved | Robert Levas | ||
|
AMBARI-8700 | Create orchestrator to manage enabling and disabling Kerberos on a cluster | Resolved | Robert Levas | ||
|
AMBARI-8705 | Kerberos wizard: API call to save krb5-conf configuration fails with server error | Resolved | Robert Levas | ||
|
AMBARI-8722 | Add method to retrieve KerberosDescriptor from AmbariMetaInfo | Resolved | Robert Levas | ||
|
AMBARI-8725 | Inject Clusters object into KerberosServerAction | Resolved | Robert Levas | ||
|
AMBARI-8734 | Create a more secure way to obtain and handle KDC administrator credentials | Resolved | Robert Levas | ||
|
AMBARI-8774 | Create Kerberos Descriptor Database Tables | Resolved | John Speidel | ||
|
AMBARI-8775 | Create Kerberos Descriptor Resource to be accessed via the REST API | Resolved | John Speidel | ||
|
AMBARI-8780 | Replace ${host} variable with relevant host in Kerberos Descriptors | Resolved | Robert Levas | ||
|
AMBARI-8790 | Fix HBase Kerberos Descriptor, HBASE_CLIENT is incorrect | Resolved | Robert Levas | ||
|
AMBARI-8795 | MapReduce service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8797 | Zookeeper service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8840 | Keytabs need to be created to include the encryption type of AES256 CTS mode with HMAC SHA1-96 | Resolved | Robert Levas | ||
|
AMBARI-8851 | Dynamically created keytab files containing keys created in an MIT KDC have the incorrect key number value | Resolved | Robert Levas | ||
|
AMBARI-8853 | Configuration keys in Kerberos descriptors should allow for variable replacement | Resolved | Robert Levas | ||
|
AMBARI-8855 | Fix Kerberos-related tasks to show friendly names in UI ops list | Resolved | Robert Levas | ||
|
AMBARI-8860 | Update Service Configurations Kerberos task fails when there is no work to do | Resolved | Robert Levas | ||
|
AMBARI-8870 | Security state check must use a temporary cache that is to be destroyed when test is complete | Resolved | Robert Levas | ||
|
AMBARI-8897 | core-site properties defined in the kerberos descriptor for knox gateway component does not take effect | Resolved | Robert Levas | ||
|
AMBARI-8899 | Knox service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-8929 | Agent is spamming logs with tracebacks of "Error while executing command \'security_status\" | Resolved | Robert Levas | ||
|
AMBARI-8935 | JobHistoryServer Fails to pass service check in Kerberized cluster | Resolved | Robert Levas | ||
|
AMBARI-8941 | Distributed keytab files have the incorrect owner and group access controls | Resolved | Robert Levas | ||
|
AMBARI-8962 | Fix unit tests in resource_management.TestSecurityCommons.TestSecurityCommons | Resolved | Robert Levas | ||
|
AMBARI-8976 | Use cluster property rather than cluster-env/security_enabled to enable or disable Kerberos | Resolved | Robert Levas | ||
|
AMBARI-9006 | Variable replacement fails for some (complicated) values in org.apache.ambari.server.state.kerberos.AbstractKerberosDescriptor#replaceVariables | Resolved | Robert Levas | ||
|
AMBARI-9007 | Identity references fail to deference for service-level references in Kerberos descriptor parser | Resolved | Robert Levas | ||
|
AMBARI-9014 | Design admin principal session expiration handling API call | Resolved | Robert Levas | ||
|
AMBARI-9030 | Remove temporary api fields for obtaining kerberos descriptors for stacks and stack services | Resolved | Robert Levas | ||
|
AMBARI-9031 | Kerberos wizard: HiveServer2 start fails | Resolved | Jaimin Jetly | ||
|
AMBARI-9033 | Kerberos wizard: MapReduce2 service check (wordcount job) fails | Resolved | Jaimin Jetly | ||
|
AMBARI-9037 | Storm service components should indicate security state | Resolved | Robert Levas | ||
|
AMBARI-9045 | Capture security state from components for use in enabling or disabling Kerberos | Resolved | Robert Levas | ||
|
AMBARI-9055 | Pass LDAP URL and Principal container DN to Active Directory operations handler | Resolved | Robert Levas | ||
|
AMBARI-9077 | Add principal type to Kerberos descriptor | Resolved | Robert Levas | ||
|
AMBARI-9121 | Get kdc_type from kerberos-env rather than krb5-conf configuration | Resolved | Robert Levas | ||
|
AMBARI-9122 | Split up principal components and realm in Kerberos descriptor | Resolved | Robert Levas | ||
|
AMBARI-9136 | Need to provide meaningful names for the stage context in Kerb API call response | Resolved | Robert Levas | ||
|
AMBARI-9149 | Test principal and keytab required for service check should be created as part of kerberos service check action | Resolved | Robert Levas | ||
|
AMBARI-9159 | Kerberos should support generation of host specific user principals | Resolved | Jaimin Jetly | ||
|
AMBARI-9170 | Principal creation for Active Directory accounts should be configurable | Resolved | Robert Levas | ||
|
AMBARI-9171 | Keytab generation should use kerberos-env/encryption_types when creating key entries | Resolved | Robert Levas | ||
|
AMBARI-9209 | Add the ability to append a random value to values in LDAP attributes when generating principals in Active Directory | Resolved | Robert Levas | ||
|
AMBARI-9228 | Ambari Server setup to install and copy JCE policy file in-place (handle both Default / Custom JDK scenarios) | Resolved | Robert Levas | ||
|
AMBARI-9244 | JAAS configuration file parser leaves trailing quote in quoted values | Resolved | Robert Levas | ||
|
AMBARI-9261 | Ensure enable/disable Kerberos logic should invoke only when state of security flag is changed | Resolved | Robert Levas | ||
|
AMBARI-9279 | MapReduce2 Service Check fails after enabling Kerberos with permission issue in local filesystem | Resolved | Robert Levas | ||
|
AMBARI-9295 | Remove toLowerCase() from userPrincipalName in default Kerberos principal create template | Resolved | Robert Levas | ||
|
AMBARI-9309 | Automate Kerberos support for AMS | Resolved | Robert Levas | ||
|
AMBARI-9317 | Kerberos: Need stdout to show info on Kerberos-related tasks | Resolved | Robert Levas | ||
|
AMBARI-9323 | Kerberos: host/<hostname>@REALM principals are created (should not be created) | Resolved | Robert Levas | ||
|
AMBARI-9324 | Kerberos: when unable to connect to KDC admin, need to inform user | Resolved | Robert Levas | ||
|
AMBARI-9357 | cluster-env/security_enabled not set to true when Kerberos is enabled in cluster | Resolved | Robert Levas | ||
|
AMBARI-9359 | Remove toLowerCase() from userPrincipalName in default Kerberos principal create template | Resolved | Robert Levas | ||
|
AMBARI-9360 | Implement unkerberize for kerberized cluster | Resolved | Robert Levas | ||
|
AMBARI-9385 | Implement Keytab regeneration | Resolved | Robert Levas | ||
|
AMBARI-9406 | Service configurations are not updated as customized in the descriptor | Resolved | Robert Levas | ||
|
AMBARI-9439 | Kerberos: Do not validate host health or maintenance state when enabling Kerberos | Resolved | Robert Levas | ||
|
AMBARI-9514 | Kerberos: Keytab regeneration not invoked when initiated via API | Resolved | Robert Levas | ||
|
AMBARI-9539 | Kerberos: Kerberos service absent in stacks lower than HDP-2.2 | Resolved | Vitaly Brodetskyi | ||
|
AMBARI-9554 | Kerberos: missing config properties after enabling Kerberos | Resolved | Robert Levas | ||
|
AMBARI-9578 | Kerberos: provide option to not generate kerb client krb5.conf | Resolved | Robert Levas | ||
|
AMBARI-9580 | Set kdc_type in kerberos-env rather than krb5-conf configuration | Resolved | Robert Levas | ||
|
AMBARI-9637 | Kerberos: Escape special characters in Distinguished Names used for queries in Active Directory | Resolved | Robert Levas | ||
|
AMBARI-9642 | Required Properties in Configure Kerberos Step | Resolved | Robert Levas | ||
|
AMBARI-9661 | Security enabling fails after ambari only upgrade | Resolved | Robert Levas | ||
|
AMBARI-9666 | Kerberos: Adding a service to a Kerberized cluster requires Kerberos-related tasks occur before INSTALL stage | Resolved | Robert Levas | ||
|
AMBARI-9702 | Zookeeper start failed after upgrading secured cluster | Resolved | Robert Levas | ||
|
AMBARI-9739 | Kerberos: regenerate keytabs not handled for all hosts | Resolved | Robert Levas | ||
|
AMBARI-9742 | Kerberos: fails when entering admin principal with blank password | Resolved | Robert Levas | ||
|
AMBARI-9743 | Storm service check failed after enabling security with existing AD | Resolved | Robert Levas | ||
|
AMBARI-9749 | Kerberos: check kerb task should delete smoke user principal | Resolved | Robert Levas | ||
|
AMBARI-9775 | Oozie failed to start in secured cluster for stacks 2.0 and 2.1 | Resolved | Robert Levas | ||
|
AMBARI-9785 | Root user has spnego (HTTP) kerberos ticket set after Kerberos is enabled, root should have no ticket. | Resolved | Robert Levas | ||
|
AMBARI-9786 | Local user mapping for hdfs headless principal not set in Kerberos descriptor | Resolved | Robert Levas | ||
|
AMBARI-9802 | Phoenix is failing on ambari-installed secure clusters | Resolved | Robert Levas | ||
|
AMBARI-9804 | Agent is spamming logs with exceptions | Resolved | Robert Levas | ||
|
AMBARI-9822 | Check Pig failed after ambari only upgrade from 1.6.0 to 2.0.0 and enabling security | Resolved | Robert Levas | ||
|
AMBARI-9840 | Warning Alert for storm after enabling security | Resolved | Jayush Luniya | ||
|
AMBARI-9852 | Kerberos: Kerberos Service Check needs to generate and destroy it's own unique identity for testing | Resolved | Robert Levas | ||
|
AMBARI-9879 | Storm supervisor process shuts down on secure Ambari cluster | Resolved | Jayush Luniya | ||
|
AMBARI-9895 | security_status test cases sporadically fail due to import of status_params module | Resolved | Robert Levas | ||
|
AMBARI-9903 | core-site.xml having wrong value for hadoop.proxyuser.HTTP.groups | Resolved | Robert Levas | ||
|
AMBARI-9907 | Storm principal is marked as a service rather than a user principal, causes issue adding hosts with Supervisor | Resolved | Robert Levas | ||
|
AMBARI-9917 | Kerberos: Add Host did not generate keytabs | Resolved | Robert Levas | ||
|
AMBARI-9937 | Ambari must support deployment on separate host | Resolved | Robert Levas | ||
|
AMBARI-9981 | Ambari storm logviewer in secure mode doesn't work | Resolved | Robert Levas | ||
|
AMBARI-9985 | Maintenance Mode Hosts do not get keytabs | Resolved | Robert Levas | ||
|
AMBARI-10016 | Kerberos: Run ambari-server using non-root causes issues with AD velocity engine | Resolved | Robert Levas | ||
|
AMBARI-10018 | Kerberos: Password generator needs to generate passwords based on rules to satisfy password policy | Resolved | Robert Levas | ||
AMBARI-10037 | Support creation of a secure cluster (w/ blueprints) | Open | Unassigned | |||
|
AMBARI-10043 | Test Kerberos Client (KERBEROS_SERVICE_CHECK) is failed after Ambari only upgrade from 1.6.0 to 2.0.0 when enable securitty (Centos 5.9) | Resolved | Robert Levas | ||
|
AMBARI-10047 | 500 error on installing kerberos clients | Resolved | Robert Levas | ||
|
AMBARI-10048 | Some of Hive principal fields doesn't have default value when enabling security | Resolved | Robert Levas | ||
|
AMBARI-10053 | The path(s) to the Kerberos utilities (kadmin, klist, etc...) should be configurable | Resolved | Robert Levas | ||
|
AMBARI-10068 | Add Host incorrectly changes webhcat properties | Resolved | Jaimin Jetly | ||
|
AMBARI-10071 | Kerberos service check doesn't work is new HTTP session is created | Resolved | Robert Levas | ||
|
AMBARI-10101 | Hive alert on secured cluster | Resolved | Robert Levas | ||
|
AMBARI-10176 | Storm service check failed after disabling security | Resolved | Robert Levas |