[HIVE-19994] Impala "drop table" fails with Hive Metastore exception

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Metastore

Environment

Hadoop distribution: CHD 5.14.2

Hive version:  1.1.0-cdh5.14.2

Impala version: 2.11.0

Kudu version: 1.6.0

 

• Assignee: Karthik Manamcheri
• Reporter: Rodion Myronov
• Created: 26/Jun/18 09:22
• Updated: 17/Nov/22 08:54
• Resolved: 29/May/19 16:33

Description

"drop table" statement in Impala shell fails with the following exception:

ImpalaRuntimeException: Error making 'dropTable' RPC to Hive Metastore: CAUSED BY: MetaException: One or more instances could not be deleted

 

Metastore log file shows that "DELETE FROM `PARTITION_KEYS` WHERE `TBL_ID`=?" statement fails because of foreign key violation (full stacktrace will be added):

Caused by: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails ("hivemetastore_emtig3vtq7qp1tiooo07sb70ud"."COLUMNS_V2", CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID") REFERENCES "CDS" ("CD_ID"))

 

The table is created and then dropped as a part of ETL process executed every hour. Most of the time it works fine, the issue is not reproducible at will.

Table creation script is:

CREATE TABLE IF NOT EXISTS price_advisor_ouput.t_switching_coef_source
{{( }}
...fields here...
PRIMARY KEY (...PK field here...)
)
PARTITION BY HASH(matrix_pcd) PARTITIONS 3
STORED AS KUDU;

 

Not sure how to approach diagnostics and fix, so any input will be really appreciated. 

Thanks in advance, 

Rodion Myronov

Issue Links

• is related to: HIVE-23756 Added more constraints to the package.jdo file
• links to: Code Review