Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
When trying to export Hive to MySQL with Sqoop, I found that using hcatalog with --update-key would error.
This command worked:
env HCAT_HOME=/srv/hive-latest/hcatalog HADOOP_USER_NAME=hadoop sqoop export -Dquery_source=sql_export -Dhive.metastore.uris=thrift://<DELETED>:9083 -Dmapred.job.queue.name=ETL-nonSLA -Dmapred.job.name=attempt --hive-home /srv/hive-latest --hcatalog-database charlesyeh --connect 'jdbc:mysql://10.16.101.24/sql_export?user=<DELETED>&password=<DELETED>&charset=utf8&connect_timeout=3&use_unicode=1' --hcatalog-table result7660208 --table test_ss
Adding only "--update-key id" at the end of the previous command made it fail. The output was:
...
2016-06-04 00:15:52,051 INFO hcat.SqoopHCatUtilities: Adding jar files under /srv/hive-latest/hcatalog/share/hcatalog/storage-handlers to distributed cache (recursively)
2016-06-04 00:15:52,051 WARN hcat.SqoopHCatUtilities: No files under /srv/hive-latest/hcatalog/share/hcatalog/storage-handlers to add to distributed cache for hcatalog job
2016-06-04 00:15:52,068 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2016-06-04 00:15:52,070 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
2016-06-04 00:15:52,070 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.sqoop.mapreduce.ExportJobBase.getFileType(ExportJobBase.java:127)
at org.apache.sqoop.mapreduce.ExportJobBase.isSequenceFiles(ExportJobBase.java:118)
at org.apache.sqoop.mapreduce.ExportJobBase.inputIsSequenceFiles(ExportJobBase.java:492)
at org.apache.sqoop.mapreduce.JdbcUpdateExportJob.getMapperClass(JdbcUpdateExportJob.java:69)
at org.apache.sqoop.mapreduce.ExportJobBase.configureMapper(ExportJobBase.java:268)
at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
at org.apache.sqoop.manager.SqlManager.updateTable(SqlManager.java:941)
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:71)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)