[SQOOP-3252] Wrong AVRO schema on import of DATE column with oraoop Connector

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.4.6
• Component/s: codegen, connectors/oracle

Environment

Hortonworks HDP 2.6.0.1

• Assignee: Unassigned
• Reporter: Johannes Mayer
• Created: 03/Nov/17 08:58
• Updated: 18/Dec/18 19:49

Description

I have found a bug, when importing a table from an oracle database using the oracle connector for hadoop.

The sqoop command looks like this:

sqoop import -Dmapreduce.job.user.classpath.first=true -Doraoop.timestamp.string=true --connect jdbc:oracle:thin:@oracle_server --username USER --password xxx --as-avrodatafile --direct --table TEST_DATE --target-dir /user/admin/test_date

The generated AVRO schema for the date field is [null, long]. This is why an error is thrown when sqoop tries to write the date string to that field. The workaround for this problem is to add a mapping for this column:

--map-column-java DATE_COL=String

But since I want to build a general solution adding a mapping for each DATE column is not an option.

The type of the generated AVRO schema for DATE Columns should be [null, string] when using the Oracle connector for Hadoop and when oraoop.timestamp.string=true.