[SQOOP-1369] Avro export ignores --columns option

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.7
• Assignee: Paul Mazak
• Reporter: Lars Francke
• Created: 11/Jul/14 14:00
• Updated: 15/Sep/15 17:52
• Resolved: 15/Sep/15 14:59

Description

In JdbcExportJob AVRO_COLUMN_TYPES_MAP is being set with the full schema of the output table. This causes the AvroExportMapper to fail with unknown fields if --columns was used to restrict the columns to export (it then tries to set a value on the generated class which doesn't exist).

There are multiple ways I can see to solve this.

• Filter the columnTypes passed on to the Mapper in JdbcExportJob.configureInputFormat
• Pass the --columns value on to the AvroExportMapper and let it ignore things that are not in there
• Let AvroExportMapper not fail when it can't set a field.

I might be able to provide a patch and I'd go with the simplest (the first one probably) if there are no objections.