Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3890

Kafka Streams: task assignment is not maintained on cluster restart or rolling restart

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.0.1
    • streams

    Description

      Currently the task assignment in TaskAssignor is not deterministic. During cluster restart or rolling restart, even though the participating worker nodes are the same, but the TaskAssignor is not able to maintain a deterministic mapping, so about 20% partitions will be reassigned which would cause state repopulation on cluster restart time.

      When the participating worker nodes are not changed, we really just want to keep the old task assignment.

      Attachments

        Issue Links

          Activity

            People

              hcai@pinterest.com Henry Cai
              hcai@pinterest.com Henry Cai
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: