[HIVE-23018] Provide a bulk API to fire multiple insert events

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 4.0.0-alpha-1
• Assignee: Vihang Karajgaonkar
• Reporter: Vihang Karajgaonkar
• Created: 12/Mar/20 18:29
• Updated: 17/Nov/22 08:51
• Resolved: 31/Mar/20 07:50

Description

Metastore provides a API to fire a listener event (currently only supports INSERT event). The problem with that API is that it only takes in one partition at a time. A typical query may insert data into multiple partitions at a time. In such a case query engines like HS2 or Impala will have to issue multiple RPCs to metastore sequentially to fire these events. This can show up as a slowdown to the user if the query engines do not return the prompt to the user until all the events are fired (In case of HS2 and Impala). It would be great if we have bulk API which takes in multiple partitions for a table so that metastore can generate many such events in one RPC.