[SQOOP-1666] Sqoop2: JobManager code on failure should call SqoopDestroyerExecutor

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Labels: code-cleanup
• Assignee: Unassigned
• Reporter: Veena Basavaraj
• Created: 04/Nov/14 06:16
• Updated: 06/Mar/15 16:46
• Resolved: 06/Mar/15 16:46

Description

There is more to just calling the destroyer.destroy method in the SqoopDestroyerExecutor

Should not the code in JobManger call this utility class?

The matching code is missing in the JobManager

 public static void executeDestroyer(boolean success, Configuration configuration, Direction direction) {
    String destroyerPropertyName, prefixPropertyName;
    switch (direction) {
      default:
      case FROM:
        destroyerPropertyName = MRJobConstants.JOB_ETL_FROM_DESTROYER;
        prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_FROM_CONTEXT;
        break;

      case TO:
        destroyerPropertyName = MRJobConstants.JOB_ETL_TO_DESTROYER;
        prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_TO_CONTEXT;
        break;
    }

    Destroyer destroyer = (Destroyer) ClassUtils.instantiate(configuration.get(destroyerPropertyName));

    if(destroyer == null) {
      LOG.info("Skipping running destroyer as non was defined.");
      return;
    }

    // Objects that should be pass to the Destroyer execution
    PrefixContext subContext = new PrefixContext(configuration, prefixPropertyName);
    Object configConnection = MRConfigurationUtils.getConnectorConnectionConfig(direction, configuration);
    Object configJob = MRConfigurationUtils.getConnectorJobConfig(direction, configuration);

    // Propagate connector schema in every case for now
    Matcher matcher = MatcherFactory.getMatcher(
        MRConfigurationUtils.getConnectorSchema(Direction.FROM, configuration),
        MRConfigurationUtils.getConnectorSchema(Direction.TO, configuration));
    Schema schema = direction == Direction.FROM ?
        matcher.getFromSchema() : matcher.getToSchema();

    DestroyerContext destroyerContext = new DestroyerContext(subContext, success, schema);

    LOG.info("Executing destroyer class " + destroyer.getClass());
    destroyer.destroy(destroyerContext, configConnection, configJob);

Issue Links

• is a clone of: SQOOP-1665 Sqoop2: Misc Cleanup / rename lingering connection to link