Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-872 Uber jira for HMS HA + Sentry HA redesign
  3. SENTRY-1669

HMSFollower should read current processed notification ID from database every time it runs

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: Hdfs Plugin
    • Labels:
      None

      Description

      Instead of using in-memory current processed notification ID, HMSFollower should read the processed notification ID info from database every time it runs. Otherwise, after failover, the new leader cannot get the correct up-to-date processed notification ID.

      It should also update the Notification ID to database once it processes a notification event. The Notification ID in database should keep on increase (we don't handle its overflow scenario).

      HMS notification ID's are stored in separate table for couple of reasons

      1. SENTRY_PATH_CHANGE is not updated for every notification that is received from HMS. There are cases where HMSFollower doesn't process notifications and skip's them. Depending on SENTRY_PATH_CHANGE information may not provide the last notification processed.
      2. There could be cases where HMSFollower thread in multiple sentry servers be acting as a leader and process HMS notifications. we need to avoid processing the notifications multiple times. This can be made sure by always having some number of notification information always regardless of purging interval.
        3. SENTRY_PATH_CHANGE information stored can typically be removed once namenode plug-in has processed the update.

      As purpose and usage of notification ID information is different from PATH update info, it locally makes sense to store notification ID separately.

        Attachments

        1. SENTRY-1669.001-sentry-ha-redesign.patch
          76 kB
          Krishna Kalyan
        2. SENTRY-1669.002-sentry-ha-redesign.patch
          46 kB
          Krishna Kalyan
        3. SENTRY-1669.003-sentry-ha-redesign.patch
          46 kB
          Krishna Kalyan
        4. SENTRY-1669.004-sentry-ha-redesign.patch
          48 kB
          Krishna Kalyan
        5. SENTRY-1669.005-sentry-ha-redesign.patch
          49 kB
          Krishna Kalyan
        6. SENTRY-1669.006-sentry-ha-redesign.patch
          49 kB
          Krishna Kalyan
        7. SENTRY-1669.007-sentry-ha-redesign.patch
          49 kB
          Krishna Kalyan
        8. SENTRY-1669.008-sentry-ha-redesign.patch
          50 kB
          Krishna Kalyan
        9. SENTRY-1669.009-sentry-ha-redesign.patch
          50 kB
          Krishna Kalyan
        10. SENTRY-1669.010-sentry-ha-redesign.patch
          50 kB
          Krishna Kalyan
        11. SENTRY-1669.011-sentry-ha-redesign.patch
          56 kB
          Krishna Kalyan
        12. SENTRY-1669.012-sentry-ha-redesign.patch
          56 kB
          Krishna Kalyan
        13. SENTRY-1669.013-sentry-ha-redesign.patch
          56 kB
          Krishna Kalyan
        14. SENTRY-1669.014-sentry-ha-redesign.patch
          50 kB
          Krishna Kalyan
        15. SENTRY-1669.016-sentry-ha-redesign.patch
          56 kB
          Krishna Kalyan

          Issue Links

            Activity

              People

              • Assignee:
                kkalyan Krishna Kalyan
                Reporter:
                hahao Hao Hao
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: