XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • Impala 4.0.0
    • None
    • None
    • ghx-label-6

    Description

      In case of INSERT_EVENTS if Impala inserts into a table it causes a refresh to the underlying table/partition. This could be unnecessary when there is only one Impala cluster in the system. The existing self-event detection framework cannot identify such events because they are not sending HMS objects like tables and partitions to the HMS. Instead in case of INSERT_EVENT HMS API only asks for a table name or partition value to fire a insert event on it. 

      We can detect a self-event in such cases if the HMS API to fire a listener event is improved to return the event id. This would be used by EventProcessor to ignore the event when it is fetched later in the next polling cycle. In order to support this, we will need to make a change to Hive as well so that the enhanced API can be used.

      Attachments

        Activity

          People

            Xiaomeng Zhang Xiaomeng Zhang
            vihangk1 Vihang Karajgaonkar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: