Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3012

Avoid reserved.broker.max.id collisions on upgrade

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.9.0.1, 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      I have seen a few issues utilizing the default value of reserved.broker.max.id=1000 in existing clusters during upgrades. For many that started broker.ids at 0 and incremented by 1 this may not be an issue. However, others may have very different numbering schemes.

      Given that we have the entire Integer range to work with, I propose we increase the default to a much larger value. This will help reduce upgrade issues and also help more clearly identify generated broker ids.

      Would defaulting to 1,000,000,000 be to large? Perhaps 1,000,000 is enough.

        Issue Links

          Activity

          Hide
          gwenshap Gwen Shapira added a comment -

          Issue resolved by pull request 762
          https://github.com/apache/kafka/pull/762

          Show
          gwenshap Gwen Shapira added a comment - Issue resolved by pull request 762 https://github.com/apache/kafka/pull/762
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/kafka/pull/762

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/762
          Hide
          granthenke Grant Henke added a comment -

          Through discussion here it has been shown that a compatible and universal change that prevents all collisions would be difficult. The patch provided instead provides users who are manually configuring their broker ids to opt out of auto generation all together. This option will also prevent an accidental uses of the auto generation feature in environments where its use is unfavorable.

          I would have liked to default it to false and make the user opt into auto generation. However, thats not a compatible change given that it is already enabled by default in 0.9.0.

          Show
          granthenke Grant Henke added a comment - Through discussion here it has been shown that a compatible and universal change that prevents all collisions would be difficult. The patch provided instead provides users who are manually configuring their broker ids to opt out of auto generation all together. This option will also prevent an accidental uses of the auto generation feature in environments where its use is unfavorable. I would have liked to default it to false and make the user opt into auto generation. However, thats not a compatible change given that it is already enabled by default in 0.9.0.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user granthenke opened a pull request:

          https://github.com/apache/kafka/pull/762

          KAFKA-3012: Avoid reserved.broker.max.id collisions on upgrade

          Provides a configuration to opt out of broker id generation.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/granthenke/kafka id-generation

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/kafka/pull/762.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #762


          commit c60449ff37664c8f05a9b3faef96b05bcd821c32
          Author: Grant Henke <granthenke@gmail.com>
          Date: 2016-01-12T22:59:55Z

          KAFKA-3012: Avoid reserved.broker.max.id collisions on upgrade

          Provides a configuration to opt out of broker id generation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/762 KAFKA-3012 : Avoid reserved.broker.max.id collisions on upgrade Provides a configuration to opt out of broker id generation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/granthenke/kafka id-generation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/762.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #762 commit c60449ff37664c8f05a9b3faef96b05bcd821c32 Author: Grant Henke <granthenke@gmail.com> Date: 2016-01-12T22:59:55Z KAFKA-3012 : Avoid reserved.broker.max.id collisions on upgrade Provides a configuration to opt out of broker id generation.

            People

            • Assignee:
              granthenke Grant Henke
              Reporter:
              granthenke Grant Henke
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development