[IMPALA-10656] Fire insert events before commit

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend, Frontend
• Epic Color: ghx-label-12
• Assignee: Csaba Ringhofer
• Reporter: Csaba Ringhofer
• Created: 13/Apr/21 17:31
• Updated: 04/Nov/22 15:05
• Resolved: 04/Nov/22 15:05

Description

Currently Impala commits an insert first, then reloads the table from HMS, and generates the insert events based on the difference between the two snapshots. (e.g. which file was not present in the old snapshot but are there in the new). Hive replication expects the insert events before the commit, so this may potentially lead to issues there,

The solution is to collect the new files during the insert in the backend, and send the insert events based on this file set.

Issue Links

• causes: IMPALA-10692 Inserting to ACID tables are broken in local_catalog mode with hms event polling