[IMPALA-12890] Concurrent Delete operations doesn't throw any error

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: impala-iceberg
• Epic Color: ghx-label-13
• Assignee: Unassigned
• Reporter: JK Pasimuthu
• Created: 11/Mar/24 19:00
• Updated: 11/Mar/24 19:02

Description

If a DELETE operation runs concurrently, there is no error thrown and the wrong results are returned in the end. 

 

Repro:

DROP TABLE IF EXISTS ice_test;
CREATE TABLE IF NOT EXISTS ice_test (id INT, j bigint) STORED BY ICEBERG TBLPROPERTIES('format-version'='2');
INSERT INTO ice_test values (0, 0), (1, 0), (2, 0), (3, 0), (4, 0), (5, 0), (6, 0), (7, 0), (8, 0), (9, 0), (10, 0), (11, 0), (12, 0), (13, 0), (14, 0), (15, 0), (16, 0), (17, 0), (18, 0), (19, 0), (20, 0), (21, 0), (22, 0), (23, 0), (24, 0), (25, 0), (26, 0), (27, 0), (28, 0), (29, 0), (30, 0), (31, 0), (32, 0), (33, 0), (34, 0), (35, 0), (36, 0), (37, 0), (38, 0), (39, 0), (40, 0), (41, 0), (42, 0), (43, 0), (44, 0), (45, 0), (46, 0), (47, 0), (48, 0), (49, 0), (50, 0), (51, 0), (52, 0), (53, 0), (54, 0), (55, 0), (56, 0), (57, 0), (58, 0), (59, 0), (60, 0), (61, 0), (62, 0), (63, 0), (64, 0), (65, 0), (66, 0), (67, 0), (68, 0), (69, 0), (70, 0), (71, 0), (72, 0), (73, 0), (74, 0), (75, 0), (76, 0), (77, 0), (78, 0), (79, 0), (80, 0), (81, 0), (82, 0), (83, 0), (84, 0), (85, 0), (86, 0), (87, 0), (88, 0), (89, 0), (90, 0), (91, 0), (92, 0), (93, 0), (94, 0), (95, 0), (96, 0), (97, 0), (98, 0), (99, 0), (100, 0);

DELETE FROM ice_test WHERE id in (SELECT MAX(id) FROM ice_test);

SELECT COUNT FROM ice_test

output: 94

 

Expected output:

Validation conflict delete file error

Issue Links

• Blocked: IMPALA-12787 Concurrent DELETE and UPDATE operations on Iceberg tables can be problematic