[SQOOP-1804] Repository changes: Add editable and override attribute to inputs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.99.5
• Assignee: Veena Basavaraj
• Reporter: Veena Basavaraj
• Created: 24/Nov/14 23:38
• Updated: 19/Feb/15 00:07
• Resolved: 05/Feb/15 16:37

Description

Details of this proposal are in the wiki.
https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Wheretostoretheoutputinsqoop?
Update: The above highlights the pros and cons of each approach.

#4 is chosen, since it is less intrusive, more clean and allows U/Edit per value in the output easily.

Will use this ticket for more detailed discussion on storage options for the output from connectors
1.

// will have FK to submission
 public static final String QUERY_CREATE_TABLE_SQ_JOB_OUTPUT_SUBMISSION =
     "CREATE TABLE " + TABLE_SQ_JOB_OUTPUT + " ("
     + COLUMN_SQ_JOB_OUT_ID + " BIGINT GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1), "
     + COLUMN_SQ_JOB_OUT_KEY + " VARCHAR(32), "
     + COLUMN_SQ_JOB_OUT_VALUE + " LONG VARCHAR,"
     + COLUMN_SQ_JOB_OUT_TYPE + " VARCHAR(32),"
     + COLUMN_SQD_ID + " VARCHAR(32)," // FK to the direction table, since this allows to distinguish output from FROM/ TO part of the job
   + COLUMN_SQRS_SUBMISSION + " BIGINT, "
   + "CONSTRAINT " + CONSTRAINT_SQRS_SQS + " "
     + "FOREIGN KEY (" + COLUMN_SQRS_SUBMISSION + ") "
       + "REFERENCES " + TABLE_SQ_SUBMISSION + "(" + COLUMN_SQS_ID + ") ON DELETE CASCADE "

2.
At the code level, we will define MOutputType, one of the types can be BLOB as well, if a connector decides to store the value as a BLOB

class JobOutput {

String key;
Object value;
MOutputType type;

}

3.
At the repository API, add a new API to get job output for a particular submission Id and allow updates on values.

Issue Links

• links to: RB