Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1689

Add validations before state transitions in ZkBarrierForVersionUpgrade.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.1
    • Component/s: None
    • Labels:
      None

      Description

      Problem:
      A barrier in zookeeper can be in either DONE or TimedOut state. Existing ZkBarrier implementation doesn't have any state validations before acting upon the barrier events received from zookeeper. This can corrupt barrier states when any of the following occur:

      • When a stream processor joins a barrier marked as timed out by the leader of the group.
      • When the leader of a group expires a barrier marked as Done. This can happen when a leader plays out a stale events from its in-memory buffer.

      This will cause the processors group to go through unnecessary rebalancing phase(worse, corrupt the barrier state).

      Changes:

      • Introduce a barrier state: NEW which will be the beginning state of a barrier(In existing implementation, barrier is created with empty/null value). Barrier will be marked as Done or TimedOut only if the current barrier state is NEW. 
      • Fix the disabled barrier test cases. Add new test cases to validate the changed functionality.
      • Improve logging to aid debugging.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                spvenkat Shanthoosh Venkataraman
                Reporter:
                spvenkat Shanthoosh Venkataraman
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: