[IMPALA-11335] WriteId must be requested before taking locks during inserts

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Component/s: Catalog, Frontend
• Labels: ACID
• Epic Color: ghx-label-13
• Assignee: Unassigned
• Reporter: Csaba Ringhofer
• Created: 02/Jun/22 14:01
• Updated: 23/Aug/22 05:50
• Resolved: 10/Jun/22 16:13

Description

It turned out that the writeId as saved to DB by HMS during lock creating for inserts, because this info is used to delete the folders created by aborted/time outed inserts. This seems a bit hacky but makes sense, as during lock creation we express the intention of the transaction for the given table ( https://github.infra.cloudera.com/CDH/hive/blob/4604ca6f1077dd808055539e95e9b9be97cdb312/standalone-metastore/src/main/thrift/hive_metastore.thrift#L1123 ), while this information is express in th other APIs (open_txns, allocate_table_write_ids).

Currently Impala takes the lock first, which can cause issues during the cleanup of aborted/timeouted inserts.