[SQOOP-2008] Matching rules between from / to schema -revisted ( Use the avro model)

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Assignee: Unassigned
• Reporter: Veena Basavaraj
• Created: 13/Jan/15 16:51
• Updated: 21/Oct/15 20:29
• Resolved: 21/Oct/15 20:29

Description

Is it a norm to fill unmatched columns with nulls as we do ?

  protected void tryFillNullInArrayForUnexpectedColumn(Column column,
      Object[] array, int index) throws SqoopException {
    if (!column.isNullable()) {
      throw new SqoopException(SchemaError.SCHEMA_0004, "Target column " +
          column + " didn't match with any source column and cannot be null.");
    }

    LOG.warn("Column " + column +
        " has no matching source column. Will be ignored.");
    array[index] = null;
  }

I looked at kites schema resolution is neat
http://kitesdk.org/docs/0.17.1/Schema-Evolution.html
similar concepts can be applied in sqoop and it relies on of how avro does reader/ writer schema resolution