[ZEPPELIN-5908] JDBCSecurityImpl should reloginFromKeytab when authType='kerberos' to void kerberos ticket expired

• Type: Wish
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Assignee: Unassigned
• Reporter: Xie Yi
• Created: 08/May/23 06:33
• Updated: 12/May/23 13:04
• Resolved: 12/May/23 13:04

Description

1. Question

for  jdbc interpreter. in some situation,  'zeppelin.jdbc.auth.type'= 'KERBEROS', such as hive

if interprete open for long time(more than 12h),  and execyte sql, it will throw exception

Could not open client transport for any of the Server URI's in ZooKeeper: GSS initiate failed

 

2. Here is the process of each  jdbc executor

org.apache.zeppelin.jdbc.JDBCInterpreter internalInterpret-> org.apache.zeppelin.jdbc.JDBCInterpreter executeSql->org.apache.zeppelin.jdbc.JDBCInterpreter getConnection->org.apache.zeppelin.jdbc.security.JDBCSecurityImpl createSecureConfiguration 

org.apache.zeppelin.jdbc.JDBCInterpreter getConnection, if  "zeppelin.jdbc.auth.type"="KERBEROS", it will  run  JDBCSecurityImpl.createSecureConfiguration(getProperties(), UserGroupInformation.AuthenticationMethod.KERBEROS);

public Connection getConnection(String dbPrefix, InterpreterContext context)
    throws ClassNotFoundException, SQLException, InterpreterException, IOException {

  if (dbPrefix == null || basePropertiesMap.get(dbPrefix) == null) {
    LOGGER.warn("No such dbPrefix: {}", dbPrefix);
    return null;
  }

  Connection connection = null;
  String user = getUser(context);
  JDBCUserConfigurations jdbcUserConfigurations = getJDBCConfiguration(user);
  setUserProperty(dbPrefix, context);

  final Properties properties = jdbcUserConfigurations.getPropertyMap(dbPrefix);
  String url = properties.getProperty(URL_KEY);
  url = appendProxyUserToURL(url, user, dbPrefix);
  String connectionUrl = appendTagsToURL(url, context);

  String authType = getProperty("zeppelin.jdbc.auth.type", "SIMPLE")
          .trim().toUpperCase();
  switch (authType) {
    case "SIMPLE":
      connection = getConnectionFromPool(connectionUrl, user, dbPrefix, properties);
      break;
// if  "zeppelin.jdbc.auth.type"="KERBEROS",will login from keytab in JDBCSecurityImpl
     case "KERBEROS":
      LOGGER.debug("Calling createSecureConfiguration(); this will do " +
          "loginUserFromKeytab() if required");
      JDBCSecurityImpl.createSecureConfiguration(getProperties(),
              UserGroupInformation.AuthenticationMethod.KERBEROS);
      LOGGER.debug("createSecureConfiguration() returned");
      boolean isProxyEnabled = Boolean.parseBoolean(
              getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable", "true"));
      if (basePropertiesMap.get(dbPrefix).containsKey("proxy.user.property")
              || !isProxyEnabled) {
        connection = getConnectionFromPool(connectionUrl, user, dbPrefix, properties);
      } else {
        UserGroupInformation ugi = null;
        try {
          ugi = UserGroupInformation.createProxyUser(
                  user, UserGroupInformation.getCurrentUser());
        } catch (Exception e) {
          LOGGER.error("Error in getCurrentUser", e);
          throw new InterpreterException("Error in getCurrentUser", e);
        }

        final String poolKey = dbPrefix;
        final String finalUser = user;
        try {
          connection = ugi.doAs((PrivilegedExceptionAction<Connection>) () ->
                  getConnectionFromPool(connectionUrl, finalUser, poolKey, properties));
        } catch (Exception e) {
          LOGGER.error("Error in doAs", e);
          throw new InterpreterException("Error in doAs", e);
        }
      }
      break;
  }

  return connection;
} 

 

 

org.apache.zeppelin.jdbc.security.JDBCSecurityImpl   createSecureConfiguration

if  authType="KERBEROS" ,it will UserGroupInformation.loginUserFromKeytab

when interpreter keeped open for more than 12h, the tgt had expirek but the code will do nothing

public static void createSecureConfiguration(Properties properties,
    AuthenticationMethod authType) {
  switch (authType) {
    case KERBEROS:
      Configuration conf = new
          org.apache.hadoop.conf.Configuration();
      conf.set("hadoop.security.authentication", KERBEROS.toString());
      UserGroupInformation.setConfiguration(conf);
      try {
        // Check TGT before calling login
        // Ref: https://github.com/apache/hadoop/blob/release-3.0.1-RC1/hadoop-common-project/
        // hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1232
        if (!UserGroupInformation.isSecurityEnabled()
            || UserGroupInformation.getCurrentUser().getAuthenticationMethod() != KERBEROS
            || !UserGroupInformation.isLoginKeytabBased()) {
          String keytab = properties.getProperty("zeppelin.jdbc.keytab.location");
          String principal = properties.getProperty("zeppelin.jdbc.principal");
          UserGroupInformation.loginUserFromKeytab(principal, keytab);
          LOGGER.info("Login successfully via keytab: {} and principal: {}", keytab, principal);
        } else {
//when interpreter keeped open for more than 12h, the tgt had expirek but the code will do nothing
           LOGGER.info("The user has already logged in using Keytab and principal, " +
              "no action required");
        }
      } catch (IOException e) {
        LOGGER.error("Failed to get either keytab location or principal name in the " +
            "interpreter", e);
      }
  }
} 

here is the log for interpreter

//when first execute sql, it will login via keytab
zeppelin-interpreter-hive-shared_process-analysis-zeppelin-solo-865f989886-4lfd4.log:2023-05-07 14:44:56,954 INFO  org.apache.zeppelin.jdbc.security.JDBCSecurityImpl           [] - Login successfully via keytab: /disk1/eadop/keytab/analysis_dev.keytab and principal: analysis/[e-mail]
//every query after that, do no action, because use has logged
zeppelin-interpreter-hive-shared_process-analysis-zeppelin-solo-865f989886-4lfd4.log:2023-05-07 15:17:59,169 INFO  org.apache.zeppelin.jdbc.security.JDBCSecurityImpl           [] - The user has already logged in using Keytab and principal, no action required 

 

after 12h, keep execute sql, it will throw exception, because tgt had expired.

Issue Links

• links to: GitHub Pull Request #4597