Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-167

Move partition assignment to the broker

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      Currently partitions are assigned to consumers for consumption via a co-ordination algorithm by the consumers. This means there is effectively no master broker node, which is nice in a way, but makes the consumer logic complex to implement. It would be good to move the co-ordination to the brokers. This would make implementation of a consumer client much much easier, since only minimal zk interaction would be required. There are a number of details that would have to be worked out to finalize the protocol. Discussion is here:

      http://mail-archives.apache.org/mod_mbox/incubator-kafka-dev/201109.mbox/%3cCAFbh0Q2ADjXcbCYqGh8TB4jXBAs3Hq5RLxk9o9w76we-bWanVw@mail.gmail.com%3e

      Attachments

        Issue Links

          duplicates

          New Feature - A new feature of the product, which has yet to be developed. KAFKA-1326 New consumer checklist

          • Major - Major loss of function.
          • Resolved
          is part of

          New Feature - A new feature of the product, which has yet to be developed. KAFKA-264 Change the consumer side load balancing and distributed co-ordination to use a consumer co-ordinator

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              jkreps Jay Kreps
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: