[HADOOP-14687] AuthenticatedURL will reuse bad/expired session cookies

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.2
• Component/s: common
• Target Version/s: 2.8.2
• Hadoop Flags: Reviewed
• Assignee: Daryn Sharp
• Reporter: Daryn Sharp
• Created: 26/Jul/17 13:22
• Updated: 16/Mar/18 02:10
• Resolved: 22/Aug/17 21:59

Description

AuthenticatedURL with kerberos was designed to perform spnego, then use a session cookie to avoid renegotiation overhead. Unfortunately the client will continue to use a cookie after it expires. Every request elicits a 401, connection closes (despite keepalive because 401 is an "error"), TGS is obtained, connection re-opened, re-requests with TGS, repeat cycle. This places a strain on the kdc and creates lots of time_wait sockets.

The main problem is unbeknownst to the auth url, the JDK transparently does spnego. The server issues a new cookie but the auth url doesn't scrape the cookie from the response because it doesn't know the JDK re-authenticated.