[HIVE-16738] Notification ID generation in DBNotification might not be unique across HS2 instances.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: HiveServer2
• Assignee: Anishek Agarwal
• Reporter: Anishek Agarwal
• Created: 23/May/17 06:52
• Updated: 22/May/18 23:58
• Resolved: 09/Aug/17 06:54

Description

Going to explain the problem in scope of "replication" feature for hive 2 that is being built, as it is easier to explain:

To allow replication to work we need to set "hive.metastore.transactional.event.listeners" to DBNotificationListener. For use cases where there are multiple HiveServer2 Instances running

 private void process(NotificationEvent event, ListenerEvent listenerEvent) throws MetaException {
    event.setMessageFormat(msgFactory.getMessageFormat());
    synchronized (NOTIFICATION_TBL_LOCK) {
      LOG.debug("DbNotificationListener: Processing : {}:{}", event.getEventId(),
          event.getMessage());
      HMSHandler.getMSForConf(hiveConf).addNotificationEvent(event);
    }

      // Set the DB_NOTIFICATION_EVENT_ID for future reference by other listeners.
      if (event.isSetEventId()) {
        listenerEvent.putParameter(
            MetaStoreEventListenerConstants.DB_NOTIFICATION_EVENT_ID_KEY_NAME,
            Long.toString(event.getEventId()));
      }
  }

the above code in DBNotificationListner having the object lock wont be guarantee enough to make sure that all events get a unique id. The transaction isolation level at the db "read-comitted" or "repeatable-read" would also not guarantee the same, unless a lock is at the db level preferably on table NOTIFICATION_SEQUENCE which only has one row.

Issue Links

• is duplicated by: HIVE-16886 HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
• is related to: SENTRY-1895 Sentry should handle the case of multiple notifications with the same ID
• is related to: HIVE-14907 Hive Metastore should use repeatable-read consistency level
• relates to: HIVE-16886 HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently