[HIVE-12087] IMPORT TABLE fails

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.2.1
• Fix Version/s: 2.0.0
• Component/s: Import/Export

Environment

Hortonworks HDP 2.3

• Assignee: Willem van Asperen Assign to me
• Reporter: Willem van Asperen
• Created: 11/Oct/15 07:18
• Updated: 16/Feb/16 23:50
• Resolved: 11/Oct/15 15:22

Description

IMPORT TABLE fails for larger tables with:

0: jdbc:hive2://hdpprdhiv01.prd.xxx:10001/> import from '/tmp/export/repository/res_sales_navigator';
INFO  : Copying data from hdfs://hdpprdmas01.prd.xxx:8020/tmp/export/repository/res_sales_navigator/valid_from=201508250000 to hdfs://hdpprdmas01.prd.xxx:8020/tmp/export/repository/res_sales_navigator/.hive-staging_hive_2015-10-07_20-55-37_456_5706704167497413401-2/-ext-10000
INFO  : Copying file: hdfs://hdpprdmas01.prd.xxx:8020/tmp/export/repository/res_sales_navigator/valid_from=201508250000/part-r-00000
ERROR : Failed with exception Cannot get DistCp constructor: org.apache.hadoop.tools.DistCp.<init>()
java.io.IOException: Cannot get DistCp constructor: org.apache.hadoop.tools.DistCp.<init>()
	at org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp(Hadoop23Shims.java:1160)
	at org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:553)
	at org.apache.hadoop.hive.ql.exec.CopyTask.execute(CopyTask.java:82)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1195)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
	at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
	at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask (state=08S01,code=1)

Issue Links

• duplicates: HIVE-11607 Export tables broken for data > 32 MB