Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12053

Expose event-processor error message in catalogd WebUI

    XMLWordPrintableJSON

Details

    Description

      Currently, we can only check logs to know why event-processor comes into ERROR state. E.g. here is the error message for IMPALA-12002:

      E0315 17:54:14.959136  2659 MetastoreEventsProcessor.java:865] Unexpected exception received while processing event
      Java exception follows:
      org.apache.impala.catalog.events.MetastoreNotificationException: EventId: 26044 EventType: COMMIT_COMPACTION_EVENT Unable to parse commit compaction message
              at org.apache.impala.catalog.events.MetastoreEvents$CommitCompactionEvent.<init>(MetastoreEvents.java:2684)
              at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEventFactory.get(MetastoreEvents.java:223)
              at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEventFactory.getFilteredEvents(MetastoreEvents.java:255)
              at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:999)
              at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:851)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.impala.catalog.DatabaseNotFoundException: Database 'test_acid_compaction_5c682e9f' not found
              at org.apache.impala.catalog.Catalog.getTable(Catalog.java:196)
              at org.apache.impala.catalog.events.MetastoreEvents$CommitCompactionEvent.<init>(MetastoreEvents.java:2679)
              ... 11 more
      E0315 17:54:14.959192  2659 MetastoreEventsProcessor.java:1040] Notification event is null
      W0315 17:54:15.959450  2659 MetastoreEventsProcessor.java:844] Event processing is skipped since status is ERROR. Last synced event id is 26042 

      It'd be helpful to expose the stacktrace in catalogd's WebUI. Tests can also retrieve it to show more detailed failures instead of just complaining timeout.

      Attachments

        1. catalog-events-webui.jpg
          304 kB
          Quanlong Huang
        2. new_events_page.jpg
          275 kB
          Quanlong Huang

        Issue Links

          Activity

            People

              stigahuang Quanlong Huang
              stigahuang Quanlong Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: