Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23018

Provide a bulk API to fire multiple insert events

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0, 4.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Metastore provides a API to fire a listener event (currently only supports INSERT event). The problem with that API is that it only takes in one partition at a time. A typical query may insert data into multiple partitions at a time. In such a case query engines like HS2 or Impala will have to issue multiple RPCs to metastore sequentially to fire these events. This can show up as a slowdown to the user if the query engines do not return the prompt to the user until all the events are fired (In case of HS2 and Impala). It would be great if we have bulk API which takes in multiple partitions for a table so that metastore can generate many such events in one RPC.

        Attachments

        1. HIVE-23018.01.patch
          764 kB
          Vihang Karajgaonkar
        2. HIVE-23018.02.patch
          764 kB
          Vihang Karajgaonkar
        3. HIVE-23018.03.patch
          764 kB
          Vihang Karajgaonkar
        4. HIVE-23018.04.patch
          764 kB
          Vihang Karajgaonkar
        5. HIVE-23018.05.branch-3.patch
          965 kB
          Vihang Karajgaonkar

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar Assign to me
              Reporter:
              vihangk1 Vihang Karajgaonkar

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment