[SQOOP-409] hive-drop-import-delims doesn't work with MySQL direct import

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0-incubating
• Fix Version/s: 1.4.1-incubating
• Component/s: hive-integration
• Assignee: Jarek Jarcec Cecho
• Reporter: Lars Francke
• Created: 14/Dec/11 14:51
• Updated: 18/Jan/12 00:55
• Resolved: 17/Jan/12 23:53

Description

When using hive-drop-import-delims together with direct on a MySQL database the delimiters are not dropped. I have not tested PostgreSQL yet.

I tried looking at the code but it wasn't easy to follow.

There seem to be at least two problems:

• The logic in MySQLDumpMapper.map only looks for MySQLUtils.outputDelimsAreMySQL(conf) to decide if it needs to be reparsed or not, this should also include the hive options
• This is just a guess at the real problem: The ReparsingStreamThread calls a RecordParser directly without using the created ORM classes and thus circumventing the calls to hiveStringDropDelims