[MAPREDUCE-3101] [Umbrella] Security issues in YARN

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Component/s: security
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 27/Sep/11 05:15
• Updated: 08/May/15 18:17
• Resolved: 08/May/15 18:17

Description

Most of the chassis for security in YARN is set up and is working. There are known bugs and security holes though. This JIRA is an umbrella ticket for tracking those.

Issue Links

• incorporates: YARN-503 DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false
• is related to: YARN-47 [Umbrella] Security issues in YARN
• relates to: BIGTOP-418 Package MAPREDUCE-2858 (MRv2 WebApp Security)

Sub-Tasks

• Remove YarnConfiguration.YARN_SECURITY_INFO: Closed
• [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources: Closed
• [MR-279] [Security] Yarn servers can't communicate with each other with hadoop.security.authorization set to true: Closed
• [MR-279] [Security] All tokens in YARN + MR should have an expiry interval: Resolved
• MR 279: Security for JobHistory service: Closed
• NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor: Closed
• ResourceManager needs to renew and cancel tokens associated with a job: Closed
• Reenable TestLinuxContainerExecutor reflecting the current NM code.: Closed
• MRv2 WebApp Security: Closed
• JobClient cannot talk to JobHistory server in secure mode: Closed
• Implement Job ACLs for MRAppMaster: Closed
• Implement Application ACLs, Queue ACLs and their interaction: Closed
• NM<->RM shared secrets should be rolled every so often.: Resolved
• [MR-279] Replace IP addresses with hostnames: Closed
• Fix log serving in NodeManager: Resolved
• Yarn+MR secure mode is broken, uncovered after MAPREDUCE-3056: Closed
• Unable to restrict users based on resourcemanager.admin.acls value set: Closed
• [MR-279] Set correct permissions for files in dist cache: Closed
• Yarn httpservers not created with access Control lists: Closed
• Authorization checks needed for AM->NM protocol: Closed
• Authorization checks needed for AM->RM protocol: Closed
• Token infrastructure for running clients which are not kerberos authenticated: Closed
• Cluster.getDelegationToken() throws NPE if client.getDelegationToken() returns null.: Closed
• Authorization of NM <=> RM with simple authentication mistakenly attempts kerberos when yarn.nodemanager.principal is defined: Resolved
• Client cannot talk to the history server in secure mode: Closed
• ContainerTokens should have an expiry interval: Closed
• Randomize master key generation for ApplicationTokenSecretManager and roll it every so often: Closed
• JobHistoryServer should store tokens to authenticate clients across restart: Resolved
• AppTokens file can/should be removed: Closed
• JobHIstoryServer should let queue admins view info of corresponding apps: Open