[KUDU-2379] Spark generates a broken authentication credentials PB

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.0, 1.7.1
• Component/s: java, spark
• Target Version/s: 1.7.1
• Code Review: https://gerrit.cloudera.org/#/c/9814/
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 27/Mar/18 03:08
• Updated: 28/Mar/18 18:23
• Resolved: 28/Mar/18 18:23

Description

KUDU-2259 introduced a regression which causes Spark to not work properly on secure clusters. The issue is the following:

• the driver calls exportAuthenticationCredentials()
  • the client hasn't yet talked to the master, so it doesn't have any credentials yet, despite having a keytab available
  • the code is as follows:

        byte[] authnData = securityContext.exportAuthenticationCredentials();
        if (authnData != null) {
          return Deferred.fromResult(authnData);
        }
    

  • previously, authnData would be null in this case, and it would fall through to connect to the cluster and then export a proper token.
  • with the new implementation, an authnData is returned which is devoid of real credentials but contains a realUser. So, it's non-null, and it gets returned immediately
• the tasks then get credentials with no tokens and can't connect