[HADOOP-12846] Credential Provider Recursive Dependencies

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Assignee: Larry McCay
• Reporter: Larry McCay
• Created: 26/Feb/16 17:37
• Updated: 30/Aug/16 01:18
• Resolved: 28/Feb/16 19:29

Description

There are a few credential provider integration points in which the use of a certain type of provider in a certain filesystem causes a recursive infinite loop.

For instance, a component such as sqoop can be protecting a db password in a credential provider within the wasb/azure filesystem. Now that HADOOP-12555 has introduced the ability to protect the access keys for wasb we suddenly need access to wasb to get the database keys which initiates the attempt to get the access keys from wasb - since there is a provider path configured for sqoop.

For such integrations, those in which it doesn't make sense to protect the access keys inside the thing that we need the keys to access, we need a solution to avoid this recursion - other than dictating what filesystems can be used by other components.

This patch proposes the ability to scrub the configured provider path of any provider types that would be incompatible with the integration point. In other words, before calling Configuration.getPassword for the access keys to wasb, we can remove any configured providers that require access to wasb.

This will require some regex expressions that can be used to identify the configuration of such provider uri's within the provider path parameter.

Issue Links

• is related to: HADOOP-13548 Remove recursive dependencies of credential providers in LdapGroupsMapping
• relates to: HADOOP-12851 S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders
• relates to: HADOOP-12548 Read s3a creds from a Credential Provider
• relates to: HADOOP-12555 WASB to read credentials from a credential provider