Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9369

Inserts on large tables could be very slow when event processing it turned on

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:
      None

      Description

      In case where large number files are being inserted into a table, the createInsertEvents method fires insert events to HMS for each partition one take a time. This could be very slow for a insert statement which is added hundreds or thousands of files.

      We should see if we can fire the insert events asynchronously instead of blocking the query from returning to the user until all the insert events are fired.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                anuragmantri Anurag Mantripragada
                Reporter:
                vihangk1 Vihang Karajgaonkar
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: