[SQOOP-3479] Incremental import on existing table in Hive should throw exception if wrong file format is used

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.4.7
• Component/s: sqoop2-client
• Assignee: Unassigned
• Reporter: Wessel Valkenburg
• Created: 23/Jul/20 09:44
• Updated: 23/Jul/20 09:44

Description

Performing an incremental import on an existing table in Hive will corrupt that table if the import is performed with a different file format (for example textfile) than existing files in the table (for example parquet). Rather than corrupting the table, sqoop could check the file format and throw an error if the formats aren't the same.

 

This happens when you manually provide flags

--incremental append --check-column [somecolumn] --last-value [somevalue]