[SQOOP-3002] Sqoop Merge Tool support composite merge-key

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.5, 1.4.6, 1.99.5, 1.99.7
• Component/s: hive-integration
• Flags: Patch
• Assignee: Unassigned
• Reporter: KaimingChen
• Created: 22/Aug/16 14:44
• Updated: 25/Oct/16 14:23

Description

When i use sqoop merge tool, i can just specify one column using --merge-key arguement.
But when my table has composite keys, i use --merge-key column1,column2 then i got an Exception:
16/08/22 15:54:15 INFO mapreduce.Job: Task Id : attempt_1470135750174_2508_m_000004_2, Status : FAILED
Error: java.io.IOException: Cannot join values on null key. Did you specify a key column that exists?
at org.apache.sqoop.mapreduce.MergeMapperBase.processRecord(MergeMapperBase.java:79)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:58)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:34)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)

Issue Links

• links to: GitHub Pull Request #26