Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6126

Reduce rebalance time by not checking if created topics are available

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: streams
    • Labels:
      None

      Description

      Within StreamPartitionAssignor#assign we create new topics and afterwards wait in an "infinite loop" until topic metadata propagated throughout the cluster. We do this, to make sure topics are available when we start processing.

      However, with this approach we "extend" the time in the rebalance phase and thus are not responsive (no calls to `poll` for liveness check and KafkaStreams#close suffers). Thus, we might want to remove this check and handle potential "topic not found" exceptions in the main thread gracefully.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mjsax Matthias J. Sax
                Reporter:
                mjsax Matthias J. Sax
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: