[RANGER-3619] REST API should return 403 when authenticated client is not allowed to access API.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0, 2.2.0
• Fix Version/s: 3.0.0, 2.3.0
• Component/s: Ranger
• Assignee: kirby zhou
• Reporter: kirby zhou
• Created: 13/Feb/22 09:37
• Updated: 04/May/22 05:49
• Resolved: 04/May/22 05:49

Description

REST API should return 403-Forbidden when authenticated client is not allowed to access API to avoid crash Ranger Clients.

 

Now, some API returns 401-Unauthorized instead of 403-Forbidden when client is already passed authentication but now allowed to do something.

In general, this will not cause any serious problems. However, there is a flaw in the SPNEGO protocol implementation of Java HTTPUrlConnection. It causes the Client to throw an unexpected exception. This will trouble the operators and developers.

 

Let me show you how it happens:

 

For example:

 

The RangerAdminClient inside KMS  want to access API "/service/secure/policies/download", but the principal is not in the allowlist.

 

1. RangerAdminClient is based on Jersey-Client
2. JerseyClient sends a HTTP-request to Ranger Service without authentication information
3. Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: Neogotiate”
4. JerseyClient sends request again with Kerberos/SPNEGO authentication tokens.
5. Tomcat/Spring inside Ranger accept the authentication, then call ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
6. ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms service”, and refuse client with 401.
7. Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: Neogotiate….” for notifying RangerAdminClient the authentication is passed.

 

Now, there is a malformed state. HTTP-status code told client authenticate is failed, but HTTP header told client authentication is passed.

 

In the RangerAdminClient side, 

 

1. sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the second 401.
2. 'inNegotiate' = true, so it is in the progress of Negotiate.
3. It checks that: if "WWW-Authenticate: Negotiate" exist then disable negotiate for following code to avoid try Negotiate once again.
4. But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
5. So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a new request header.
6. Wow, Null exception happens.
7. Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"
8. Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error is : java.lang.RuntimeException: java.lang.NullPointerException"

 

This log makes admin confused.

 

 

//ServiceRest::getServicePoliciesIfUpdated

if (isAllowed) {
//...
} else {
   httpCode = HttpServletResponse.SC_UNAUTHORIZED;
}
 

// sun.net.www.protocol.http.HttpURLConnection.getInputStream0()

// Read comments labeled "Failed Negotiate" for details.
boolean dontUseNegotiate = false;
Iterator<String> iter = responses.multiValueIterator("WWW-Authenticate");
while (iter.hasNext()) {
    String value = iter.next().trim();
    if (value.equalsIgnoreCase("Negotiate") ||
            value.equalsIgnoreCase("Kerberos")) {
        if (!inNegotiate) {
            inNegotiate = true;
        } else {
            dontUseNegotiate = true;
            doingNTLM2ndStage = false;
            serverAuthentication = null;
        }
        break;
    }
}

/**
 * Failed Negotiate
 *
 * In some cases, the Negotiate auth is supported for the
 * remote host but the negotiate process still fails (For
 * example, if the web page is located on a backend server
 * and delegation is needed but fails). The authentication
 * process will start again, and we need to detect this
 * kind of failure and do proper fallback (say, to NTLM).
 *
 * In order to achieve this, the inNegotiate flag is set
 * when the first negotiate challenge is met (and reset
 * if authentication is finished). If a fresh new negotiate
 * challenge (no parameter) is found while inNegotiate is
 * set, we know there's a failed auth attempt recently.
 * Here we'll ignore the header line so that fallback
 * can be practiced.
 *
 * inNegotiateProxy is for proxy authentication.
 */