[IMPALA-9923] Data loading of TPC-DS ORC fails with "Fail to get checksum"

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Infrastructure
• Labels: broken-build flaky
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-5
• Assignee: Zoltán Borók-Nagy
• Reporter: Tim Armstrong
• Created: 07/Jul/20 15:38
• Updated: 21/Dec/23 17:47
• Resolved: 01/Dec/20 14:23

Description

INFO  : Loading data to table tpcds_orc_def.store_sales partition (ss_sold_date_sk=null) from hdfs://localhost:20500/test-warehouse/managed/tpcds.store_sales_orc_def
INFO  : 

ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get checksum, since file /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version is under construction.
INFO  : Completed executing command(queryId=ubuntu_20200707055650_a1958916-1e85-4db5-b1bc-cc63d80b3537); Time taken: 14.512 seconds
INFO  : OK
Error: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get checksum, since file /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version is under construction. (state=08S01,code=1)
java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get checksum, since file /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version is under construction.
	at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
	at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
	at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
	at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
	at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
	at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1497)
	at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1355)
	at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:1329)
	at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1127)
	at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
	at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
	at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Closing: 0: jdbc:hive2://localhost:11050/default;auth=none

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11223/

Issue Links

• is related to: HIVE-24145 Fix preemption issues in reducers and file sink operators
• relates to: IMPALA-10376 Data loading of a functional-query ORC table fails with "Fail to get checksum"
• relates to: IMPALA-10344 Error loading functional_orc_def.alltypesaggmultifiles in hive