Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8968

Fix self-event detection on database events.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-3

      Description

      When event processing is turned on, self-events are not detected for DATABASE level events (create/alter/drop database). Reproduced using the below statements:

      CREATE DATABASE test;
      
      CREATE FUNCTION test.fn_test (INT, STRING) RETURNS BOOLEAN LOCATION '/test-warehouse/dummy.jar' SYMBOL='com.cloudera.impala.TestUdf';
      
      DROP FUNCTION test.fn_test(INT, STRING);
      
      DROP DATABASE test CASCADE; 

      Since the events processor could not detect self-events, it will try to process the ALTER_DATABASE event created by dropping the function. However, it doesn't find the DB and errors out like below:

      I0923 16:09:46.042317  6727 MetastoreEventsProcessor.java:480] Received 2 events. Start event id : 30077
      I0923 16:09:46.042317  6727 MetastoreEventsProcessor.java:480] Received 2 events. Start event id : 30077
      I0923 16:09:46.042853  6727 MetastoreEvents.java:382] EventId: 30078 EventType: ALTER_DATABASE Creating event 30078 of type ALTER_DATABASE on database test
      I0923 16:09:46.044441  6727 MetastoreEvents.java:382] EventId: 30079 EventType: DROP_DATABASE Creating event 30079 of type DROP_DATABASE on database test
      I0923 16:09:46.050657  6727 MetastoreEvents.java:231] Total number of events received: 2 Total number of events filtered out: 0
      I0923 16:09:46.051167  6727 MetastoreEvents.java:382] EventId: 30078 EventType: ALTER_DATABASE Received exception Database test not found. Ignoring self-event evaluation
      E0923 16:09:46.056273  6727 MetastoreEventsProcessor.java:525] Unexpected exception received while processing event
      Java exception follows:org.apache.impala.catalog.events.MetastoreNotificationException: Unable to process event 30078 of type ALTER_DATABASE. Event processing will be stopped.
       at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:614)
       at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:511)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.impala.catalog.DatabaseNotFoundException: Database test not found
       at org.apache.impala.catalog.CatalogServiceCatalog.updateDb(CatalogServiceCatalog.java:1060)
       at org.apache.impala.catalog.events.MetastoreEvents$AlterDatabaseEvent.process(MetastoreEvents.java:1225)
       at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:316)
       at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:609)
       ... 8 more
      E0923 16:09:46.056489  6727 MetastoreEventsProcessor.java:631] Notification event is null
      W0923 16:09:56.057376  6727 MetastoreEventsProcessor.java:504] Event processing is skipped since status is ERROR. Last synced event id is 30077

       

       

        Attachments

          Activity

            People

            • Assignee:
              anuragmantri Anurag Mantripragada
              Reporter:
              anuragmantri Anurag Mantripragada
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: