Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Currently Sqoop export job runs in insert mode (default) or update mode (with --update-key option). When in insert mode, all data are inserted and when in update mode, all data are updated in existing table. This leaves out the use case where some data may need to be updated while the rest needs to be inserted.
It also causes problems
- When in insert mode, new data may cause constraint violations if they exist already.
- When in update mode, it could result in silent dropping of records that do not match the update key.
The idea is to introduce a new "upsert" mode to update records if they exist in table already or to insert records if they do not exist yet.