[IMPALA-10507] Workaround for sporadic TPC-DS testdata insert error

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Abandoned
• Affects Version/s: Impala 3.4.0
• Component/s: Infrastructure
• Epic Color: ghx-label-3
• Assignee: Kurt Deschler
• Reporter: Kurt Deschler
• Created: 13/Feb/21 16:37
• Updated: 17/Feb/21 18:20
• Resolved: 17/Feb/21 18:20

Description

A TPC-DS insert statement generated from testdata/datasets/tpcds/tpcds_schema_template.sql fails with Tez errors trying to allocate space. This Jira is to woraround the error in the test.

Statment:

insert overwrite table store_sales partition(ss_sold_date_sk)
select ... from store_sales_unpartitioned
WHERE 2451272 <= ss_sold_date_sk and ss_sold_date_sk < 2451728
distribute by ss_sold_date_sk

Error:

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /test-warehouse/tpcds.store_sales/.hive-staging_hive_2021-01-04_11-08-20_683_8822236846070344153-996/_task_tmp.-ext-10002/ss_sold_date_sk=2451574/_tmp.000000_1 could only be written to 0 of the 1 minReplication nodes. There are 3 datanode(s) running and 3 node(s) are excluded in this operation.

Issue Links

• Blocked: IMPALA-9777 Reduce the diskspace requirements of loading the text version of tpcds.store_sales