[SQOOP-2438] Use Class.cast when creating HiveConf object in ParquetJob

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.7
• Fix Version/s: 1.5.0
• Component/s: hive-integration
• Labels: newbie
• Assignee: Unassigned
• Reporter: Abraham Elmahrek
• Created: 29/Jul/15 00:52
• Updated: 30/Oct/17 14:06

Description

To create an instance of HiveMetaStoreClient, Sqoop needs to create an instance of the HiveConf class. That class is difficult to pass around, so we have specific code in the ParquetJob.addHiveDelegationToken method to create an instance of that class specifically. Using Class<?>.cast(...), we should be able to achieve the same result, with less code.

// Need to use reflection since there's no compile time dependency on the client libs.
    Class<?> HiveConfClass;
    Class<?> HiveMetaStoreClientClass;

    try {
      HiveMetaStoreClientClass = Class.forName(HIVE_METASTORE_CLIENT_CLASS);
    } catch (ClassNotFoundException ex) {
      LOG.error("Could not load " + HIVE_METASTORE_CLIENT_CLASS
          + " when adding hive delegation token. "
          + "Make sure HIVE_CONF_DIR is set correctly.", ex);
      throw new RuntimeException("Couldn't fetch delegation token.", ex);
    }

    try {
      HiveConfClass = Class.forName(HiveConfig.HIVE_CONF_CLASS);
    } catch (ClassNotFoundException ex) {
      LOG.error("Could not load " + HiveConfig.HIVE_CONF_CLASS
          + " when adding hive delegation token."
          + " Make sure HIVE_CONF_DIR is set correctly.", ex);
      throw new RuntimeException("Couldn't fetch delegation token.", ex);
    }

    try {
      Object client = HiveMetaStoreClientClass.getConstructor(HiveConfClass).newInstance(
          HiveConfClass.getConstructor(Configuration.class, Class.class).newInstance(conf, Configuration.class)
      );
      // getDelegationToken(String kerberosPrincial)
      Method getDelegationTokenMethod = HiveMetaStoreClientClass.getMethod("getDelegationToken", String.class);
      Object tokenStringForm = getDelegationTokenMethod.invoke(client, UserGroupInformation.getLoginUser().getShortUserName());

      // Load token
      Token<DelegationTokenIdentifier> metastoreToken = new Token<DelegationTokenIdentifier>();
      metastoreToken.decodeFromUrlString(tokenStringForm.toString());
      conf.getCredentials().addToken(new Text(HIVE_METASTORE_TOKEN_ALIAS), metastoreToken);

      LOG.debug("Successfully fetched hive metastore delegation token. " + metastoreToken);
    } catch (Exception ex) {
      LOG.error("Couldn't fetch delegation token.", ex);
      throw new RuntimeException("Couldn't fetch delegation token.", ex);
    }