[IMPALA-12647] Add Hive compatible way to get modified row count in DMLs (HS2)

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Clients
• Epic Color: ghx-label-2
• Assignee: Noemi Pap-Takacs
• Reporter: Csaba Ringhofer
• Created: 18/Dec/23 10:31
• Updated: 16/Aug/24 10:24

Description

e,.g after
insert into t values (1);
print "modified 1 rows(s)"

Hive and Impala implemented this in incompatible ways using different HS2 "dialects":

• HIVE-14388 added support using TGetOperationStatusResp.numModifiedRows
• IMPALA-7290 added support using TCloseImpalaOperationResp,TDmlResult

https://github.com/apache/hive/blob/fd92b3926393f0366b87cd55d5a0ad27968f18db/service-rpc/if/TCLIService.thrift#L1120
https://github.com/apache/impala/blob/4114fe8db6ec80b2e1679e946555f91ab7043f2e/common/thrift/ImpalaService.thrift#L966

The Impala patch is newer (probably we didn't know about the Hive solution?), on the other side it is based on a much older solution in Beeswax. The Impala solution is also more "advanced" and contains extra information relevant in Kudu upserts/inserts.

Currently impala-shell uses the Impala solution while in Hive compatible strict HS2 mode it doesn't return modified row count.

impyla doesn't support modified row count: https://github.com/cloudera/impyla/issues/302
There is an extension function that parses Kudu related row counts from the profile:
https://github.com/cloudera/impyla/blob/76f0ba3221e1ff26037e36afbe4a5591168157ce/impala/hiveserver2.py#L205

Ideally there would be a solution supported by both components and clients wouldn't need to adapt to specific dialects.

Issue Links

• is blocked by: IMPALA-12696 Update to a newer version of TCLIService.thrift
• relates to: IMPALA-11095 impala-shell strict hs2 mode insert is broken
• relates to: IMPALA-7290 Move impala-shell to use HS2
• relates to: HIVE-14388 Add number of rows inserted message after insert command in Beeline