[HADOOP-11467] KerberosAuthenticator can connect to a non-secure cluster

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.7.0
• Component/s: security
• Target Version/s: 2.7.0
• Hadoop Flags: Reviewed
• Assignee: Yongjun Zhang
• Reporter: Robert Kanter
• Created: 07/Jan/15 21:19
• Updated: 27/Jul/15 20:19
• Resolved: 13/Feb/15 22:19

Description

While looking at HADOOP-10895, we discovered that the KerberosAuthenticator can authenticate with a non-secure cluster, even without falling back.

The problematic code is here:

      if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {    // <----- A
        LOG.debug("JDK performed authentication on our behalf.");
        // If the JDK already did the SPNEGO back-and-forth for
        // us, just pull out the token.
        AuthenticatedURL.extractToken(conn, token);
        return;
      } else if (isNegotiate()) {                                   // <----- B
        LOG.debug("Performing our own SPNEGO sequence.");
        doSpnegoSequence(token);
      } else {                                                      // <----- C
        LOG.debug("Using fallback authenticator sequence.");
        Authenticator auth = getFallBackAuthenticator();
        // Make sure that the fall back authenticator have the same
        // ConnectionConfigurator, since the method might be overridden.
        // Otherwise the fall back authenticator might not have the information
        // to make the connection (e.g., SSL certificates)
        auth.setConnectionConfigurator(connConfigurator);
        auth.authenticate(url, token);
      }
    }

Sometimes the JVM does the SPNEGO for us, and path A is used. However, if the KerberosAuthenticator tries to talk to a non-secure cluster, path A also succeeds in this case.
More details can be found in this comment:
https://issues.apache.org/jira/browse/HADOOP-10895?focusedCommentId=14247476&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14247476

We've actually dealt with this before. HADOOP-8883 tried to fix a related problem by adding another condition to path A that would look for a header. However, the JVM hides this header, making path A never occur. We reverted this change in HADOOP-10078, and didn't realize that there was still a problem until now.

Issue Links

• is related to: HADOOP-8883 Anonymous fallback in KerberosAuthenticator is broken
• is related to: HADOOP-10078 KerberosAuthenticator always does SPNEGO
• relates to: HADOOP-10895 HTTP KerberosAuthenticator fallback should have a flag to disable it
• relates to: OOZIE-2315 TestOozieCLI.testshareLibUpdate_withSecurity fails with Hadoop 2