[IMPALA-11588] lastProcessedEvent is not updated in MetastoreEventsProcessor.processEvents()

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Catalog
• Epic Link: event-processor-supportability
• Epic Color: ghx-label-4
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 16/Sep/22 02:45
• Updated: 18/Jul/23 00:03
• Resolved: 18/Jul/23 00:03

Description

Code snipper:

  public void processEvents() {
    NotificationEvent lastProcessedEvent = null;
    try {
      EventProcessorStatus currentStatus = eventProcessorStatus_;
      if (currentStatus != EventProcessorStatus.ACTIVE) {
        LOG.warn(String.format(
            "Event processing is skipped since status is %s. Last synced event id is %d",
            currentStatus, lastSyncedEventId_.get()));
        return;
      }

      List<NotificationEvent> events = getNextMetastoreEvents();
      processEvents(events);
    } catch (MetastoreNotificationFetchException ex) {
      // No need to change the EventProcessor state to error since we want the
      // EventProcessor to continue getting new events after HMS is back up.
      LOG.error("Unable to fetch the next batch of metastore events. Hive Metastore " +
        "may be unavailable. Will retry.", ex);
    } catch(MetastoreNotificationNeedsInvalidateException ex) {
      updateStatus(EventProcessorStatus.NEEDS_INVALIDATE);
      LOG.error("Event processing needs a invalidate command to resolve the state", ex);
    } catch (Exception ex) {
      // There are lot of Preconditions which can throw RuntimeExceptions when we
      // process events this catch all exception block is needed so that the scheduler
      // thread does not die silently
      updateStatus(EventProcessorStatus.ERROR);
      LOG.error("Unexpected exception received while processing event", ex);
      dumpEventInfoToLog(lastProcessedEvent);
    }
  }

https://github.com/apache/impala/blob/cfd79b40beab86f08ad72e0bea41eabf736d0a99/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L838-L865

"lastProcessedEvent" is a local variable and never updated so it's always null.