[SQOOP-1604] Base/ Marker class for Config and Configuration

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.99.4
• Fix Version/s: 1.99.5
• Assignee: Veena Basavaraj
• Reporter: Veena Basavaraj
• Created: 22/Oct/14 20:20
• Updated: 13/Dec/14 17:19
• Resolved: 13/Dec/14 17:19

Description

HDFS and Hbase connector code did not seem to have much functionality to in the initializer and destroyer, so provide a common empty class that connector developers can use if they have nothing to override.

It is pretty much difficult to have a empty initializer and destroyer classes with the current Initializer api

I wish our Initializer api was more like the below that enforced the typesafety much harder.

/**
 * This allows connector to define initialization work for execution,
 * for example, context configuration.
 */
public abstract class Initializer<L extends LinkConfiguration, J extends JobConfiguration> {

  /**
   * Initialize new submission based on given configuration properties. Any
   * needed temporary values might be saved to context object and they will be
   * promoted to all other part of the workflow automatically.
   *
   * @param context Initializer context object
   * @param linkConfiguration link configuration object
   * @param jobConfiguration job configuration object for the FROM and TO
   *        In case of the FROM initializer this will represent the FROM job configuration
   *        In case of the TO initializer this will represent the TO job configuration
   */
  public abstract void initialize(InitializerContext context, L linkConfiguration,
      J jobConfiguration);

We could have a base marker classes LinkConfiguration / JobConfiguration, FromJobConfiguration, ToConfiguration instead of the annotations, this way code like this would be more type safe rather than using object that can lead to a lot of class cast exception at runt time

    // link config for the FROM part of the job
    Object fromLinkConfig = ClassUtils.instantiate(fromConnector.getLinkConfigurationClass());
    ConfigUtils.fromConfigs(fromConnection.getConnectorLinkConfig().getConfigs(), fromLinkConfig);

    // link config for the TO part of the job
    Object toLinkConfig = ClassUtils.instantiate(toConnector.getLinkConfigurationClass());
    ConfigUtils.fromConfigs(toConnection.getConnectorLinkConfig().getConfigs(), toLinkConfig);

 public Object getConnectorLinkConfig(Direction type) {
    switch(type) {
      case FROM:
        return fromConnectorLinkConfig;

      case TO:
        return toConnectorLinkConfig;

      default:
        throw new SqoopException(DirectionError.DIRECTION_0000, "Direction: " + type);
    }
  }

  public void setConnectorLinkConfig(Direction type, Object config) {
    switch(type) {
      case FROM:
        fromConnectorLinkConfig = config;
        break;
      case TO:
        toConnectorLinkConfig = config;
        break;
      default:
        throw new SqoopException(DirectionError.DIRECTION_0000, "Direction: " + type);
    }
  }
  /**
   * All configuration objects
   */
  Object fromConnectorLinkConfig;
  Object toConnectorLinkConfig;

  Object fromConfig;
  Object toConfig;

  Object d
 

and instead of generic Class, we could have typed Class
before

  /**
   * @return Get link configuration group class
   */
  @SuppressWarnings("rawtypes")
  public abstract Class getLinkConfigurationClass();

after

 /**
   * @return Get link configuration group class
   */
  public abstract LinkConfiguration getLinkConfigurationClass();

Issue Links

• is a clone of: SQOOP-1554 Sqoop2: Add NullConfigurationClass/ EmptyConfigurationClass to support use cases that do not have a particular type of config