[HADOOP-12082] Support multiple authentication schemes via AuthenticationFilter

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 3.0.0-alpha2
• Component/s: security
• Assignee: Hrishikesh Gadre
• Reporter: Hrishikesh Gadre
• Created: 10/Jun/15 17:55
• Updated: 27/Mar/23 06:24
• Resolved: 25/Oct/16 15:33

Description

The requirement is to support LDAP based authentication scheme via Hadoop AuthenticationFilter. HADOOP-9054 added a support to plug-in custom authentication scheme (in addition to Kerberos) via AltKerberosAuthenticationHandler class. But it is based on selecting the authentication mechanism based on User-Agent HTTP header which does not conform to HTTP protocol semantics.

As per RFC-2616

• HTTP protocol provides a simple challenge-response authentication mechanism that can be used by a server to challenge a client request and by a client to provide the necessary authentication information.
• This mechanism is initiated by server sending the 401 (Authenticate) response with ‘WWW-Authenticate’ header which includes at least one challenge that indicates the authentication scheme(s) and parameters applicable to the Request-URI.
• In case server supports multiple authentication schemes, it may return multiple challenges with a 401 (Authenticate) response, and each challenge may use a different auth-scheme.
• A user agent MUST choose to use the strongest auth-scheme it understands and request credentials from the user based upon that challenge.

The existing Hadoop authentication filter implementation supports Kerberos authentication scheme and uses ‘Negotiate’ as the challenge as part of ‘WWW-Authenticate’ response header. As per the following documentation, ‘Negotiate’ challenge scheme is only applicable to Kerberos (and Windows NTLM) authentication schemes.
SPNEGO-based Kerberos and NTLM HTTP Authentication
Understanding HTTP Authentication

On the other hand for LDAP authentication, typically ‘Basic’ authentication scheme is used (Note TLS is mandatory with Basic authentication scheme).
http://httpd.apache.org/docs/trunk/mod/mod_authnz_ldap.html

Hence for this feature, the idea would be to provide a custom implementation of Hadoop AuthenticationHandler and Authenticator interfaces which would support both schemes - Kerberos (via Negotiate auth challenge) and LDAP (via Basic auth challenge). During the authentication phase, it would send both the challenges and let client pick the appropriate one. If client responds with an ‘Authorization’ header tagged with ‘Negotiate’ - it will use Kerberos authentication. If client responds with an ‘Authorization’ header tagged with ‘Basic’ - it will use LDAP authentication.

Note - some HTTP clients (e.g. curl or Apache Http Java client) need to be configured to use one scheme over the other e.g.

• curl tool supports option to use either Kerberos (via --negotiate flag) or username/password based authentication (via --basic and -u flags).
• Apache HttpClient library can be configured to use specific authentication scheme.
  http://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html

Typically web browsers automatically choose an authentication scheme based on a notion of “strength” of security. e.g. take a look at the design of Chrome browser for HTTP authentication

Issue Links

• is related to: HBASE-27693 Support for Hadoop's LDAP Authentication mechanism (Web UI only)
• is related to: HADOOP-14955 Document the support of multiple authentications for HTTP
• relates to: YARN-4006 YARN AltKerberos HTTP Authentication doesn't work