[IMPALA-10465] Improve Kudu DML error logging in Impala

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.2.0
• Epic Color: ghx-label-13
• Assignee: Riza Suminto
• Reporter: Grant Henke
• Created: 29/Jan/21 15:34
• Updated: 28/May/22 00:40
• Resolved: 26/May/22 14:27

Description

Kudu recently added support for INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE to handle cases where users want to ignore primary key errors in an efficient way. Impala already does this today for it's INSERT behavior, however it does so by ignoring the per-row errors from Kudu client side. This requires a large error buffer (which may need to be expanded in rare cases) to log all of the warning messages which users often do not care about and causes significant RPC overhead.

Instead it would be good to add a property or special IGNORE keyword to leverage the new ignore operations.

Issue Links

• relates to: IMPALA-7007 Kudu duplicate key count can be off
• relates to: IMPALA-7015 Insert into Kudu table returns with Status OK even if there are Kudu errors
• relates to: IMPALA-3710 INSERT, UPDATE, DELETE should ignore conflicts by default
• relates to: IMPALA-4591 Kudu client error memory should be bounded
• relates to: KUDU-1563 Add support for INSERT/UPDATE/DELETE IGNORE
• relates to: KUDU-3211 Add a cluster supported features request
• relates to: KUDU-3351 Add insert error count metrics in WriteResponsePB