Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3890

Kafka Streams: task assignment is not maintained on cluster restart or rolling restart

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.1
    • Component/s: streams
    • Labels:

      Description

      Currently the task assignment in TaskAssignor is not deterministic. During cluster restart or rolling restart, even though the participating worker nodes are the same, but the TaskAssignor is not able to maintain a deterministic mapping, so about 20% partitions will be reassigned which would cause state repopulation on cluster restart time.

      When the participating worker nodes are not changed, we really just want to keep the old task assignment.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hcai@pinterest.com Henry Cai
                Reporter:
                hcai@pinterest.com Henry Cai
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: