[SQOOP-1349] Sqoop2: Use configurable writable to get Intermediate Data Format

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.99.3
• Fix Version/s: 1.99.5
• Component/s: sqoop2-framework
• Assignee: Veena Basavaraj
• Reporter: Abraham Elmahrek
• Created: 30/Jun/14 18:30
• Updated: 13/Nov/14 20:18
• Resolved: 13/Nov/14 18:48

Description

The below code performs an unnecessary string copy.

toDataFormat.setObjectData( matcher.getMatchingData( fromDataFormat.getObjectData() ) );
writable.setString(toDataFormat.getTextData());
context.write(writable, NullWritable.get());

It would be better to just pass IDFs around.

Issue Links

• depends upon: HADOOP-10686 Writables are not always configured
• depends upon: SQOOP-777 Sqoop2: Implement intermediate data format representation policy
• is blocked by: SQOOP-1452 Sqoop2: Upgrade to Hadoop 2.5.1
• links to: RB