[SQOOP-1423] hcatalog export with --map-column-java fails

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.4, 1.4.6
• Fix Version/s: 1.4.6
• Labels: easyfix newbie
• Assignee: Muhammad Ehsan ul Haque
• Reporter: Muhammad Ehsan ul Haque
• Created: 08/Aug/14 10:51
• Updated: 09/Aug/14 19:00
• Resolved: 09/Aug/14 17:42

Description

hcat export does not use mappings provided using --map-column-java and crashes with NPE when target table has a non standard sql type.

Test when exporting to postgres

Hive table schema

CREATE TABLE hive_table(col1 String);

Hive table data

127.0.0.1
182.27.181.11
...

Postgres Table Schema

CREATE TABLE postgres_table (col1 inet)

Sqoop Command for Exporting

sqoop export --verbose --connect jdbc:postgresql://pg_server:5432/pg_db?stringtype=unspecified --username user --password xxxxx --table postgres_table --hcatalog-table hive_table --map-column-java col1=String

exception

14/08/08 12:51:04 INFO hcat.SqoopHCatUtilities: Configuring HCatalog for export job
14/08/08 12:51:04 INFO hcat.SqoopHCatUtilities: Ignoring configuration request for HCatalog info
14/08/08 12:51:04 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
	at org.apache.hadoop.io.Text.encode(Text.java:432)
	at org.apache.hadoop.io.Text.set(Text.java:196)
	at org.apache.hadoop.io.Text.<init>(Text.java:88)
	at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureExportInputFormat(SqoopHCatUtilities.java:708)
	at org.apache.sqoop.mapreduce.JdbcExportJob.configureInputFormat(JdbcExportJob.java:70)
	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:387)
	at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:828)
	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:238)