[SQOOP-824] Sqoop code generation in 'update' export mode incompatible with '--columns' option

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.2
• Fix Version/s: 1.4.3
• Component/s: codegen, tools
• Labels: manager

Environment

$ uname -a
Linux precise32 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012 i686 i686 i386 GNU/Linux

$ hadoop version
Hadoop 0.23.4
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.4/hadoop-common-project/hadoop-common -r 1392631
Compiled by evans on Mon Oct 22 13:02:13 CDT 2012
From source with checksum d5a2b33744f6dff6eecf84786baeb80c

Sqoop downloaded from http://www.eu.apache.org/dist/sqoop/1.4.2/sqoop-1.4.2.bin__hadoop-0.23.tar.gz

Running in Vagrant VM with Ubuntu 12.04 LTS 32-bit.

• Assignee: Jarek Jarcec Cecho
• Reporter: Stan Angeloff
• Created: 10/Jan/13 09:41
• Updated: 17/Oct/14 11:39
• Resolved: 20/Jan/13 15:41

Description

When using sqoop export with --update-key and --columns the options are incompatible. For example, given table T1

  +------+
  | T1   |
  +------+
  | id   |
  | name |
  | mail |
  | cell |
  +-------

When using:

$ sqoop export [ .. ] --update-key id --update-mode allowinsert --columns id,name

the code generation utility creates a Java file which fails to compile.

This seems to happen because ConnManager appends all table columns, even those not mentioned on the command-line (see ConnManager.configureDbOutputColumns). As a result, the generated class contains a write method which references class members which do not exist.

I could suggest two possible solutions: 1) either make this case an exception or 2) make the code that appends all remaining table columns optional.

I may be missing something as that code is there for a reason, but I can't see why?

Issue Links

• links to: Review board