[YARN-9334] YARN Service Client does not work with SPNEGO when knox is configured

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.2.0
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: yarn-native-services
• Assignee: Billie Rinaldi
• Reporter: Tarun Parimi
• Created: 27/Feb/19 05:06
• Updated: 28/Feb/19 18:33
• Resolved: 27/Feb/19 23:52

Description

When knox is configured, the configuration hadoop.http.authentication.type is set to org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler instead of kerberos.

We have the following check in ApiServiceClient#getApiClient for kerberos.

if (conf.get("hadoop.http.authentication.type").equals("kerberos")) {
      try {
        URI url = new URI(requestPath);
        String challenge = YarnClientUtils.generateToken(url.getHost());
        builder.header(HttpHeaders.AUTHORIZATION, "Negotiate " + challenge);
      } catch (Exception e) {
        throw new IOException(e);
      }
    }

So we always get 401 error since there is no auth handled for knox.