Kafka
  1. Kafka
  2. KAFKA-762

Improve second replica assignment

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: core
    • Labels:

      Description

      Currently, for partitions from different topics, if the first replica is assigned to the same broker, their second replica is also assigned to the same broker. This means that when a broker fails, many leaders on the failed broker will be moved to the same broker, and thus introduces more load on that broker.

        Activity

        Neha Narkhede made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Guozhang Wang added a comment -

        I think that as long as the secondShift is different for two partition, even if their first replica is assigned to the same broker their following replicas would not be possibly assigned to the same broker?

        Show
        Guozhang Wang added a comment - I think that as long as the secondShift is different for two partition, even if their first replica is assigned to the same broker their following replicas would not be possibly assigned to the same broker?
        Hide
        Jun Rao added a comment -

        Yes, the logic is a bit cryptic. I will add some more comments.

        Show
        Jun Rao added a comment - Yes, the logic is a bit cryptic. I will add some more comments.
        Hide
        Neha Narkhede added a comment -

        There is a lot of confusing index flipping logic there, do you mind adding some docs to explain that ? For example, its still unclear whether the second replica is assigned to an adjacent broker compared to first replica

        Show
        Neha Narkhede added a comment - There is a lot of confusing index flipping logic there, do you mind adding some docs to explain that ? For example, its still unclear whether the second replica is assigned to an adjacent broker compared to first replica
        Hide
        Jun Rao added a comment -

        It's still valid since the change only affects the partition assignment of different topics.

        Show
        Jun Rao added a comment - It's still valid since the change only affects the partition assignment of different topics.
        Hide
        Neha Narkhede added a comment -

        Is the partition assignment mentioned in the comment above assignReplicasToBrokers valid after this change ?

        Show
        Neha Narkhede added a comment - Is the partition assignment mentioned in the comment above assignReplicasToBrokers valid after this change ?
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.8 [ 12317244 ]
        Resolution Fixed [ 1 ]
        Hide
        Jun Rao added a comment -

        Thanks for the review. Committed to 0.8.

        The issue is that for partitions from different topics whose first replica is assigned to the same broker, without this patch, their second replica is assigned to the same broker too.

        Show
        Jun Rao added a comment - Thanks for the review. Committed to 0.8. The issue is that for partitions from different topics whose first replica is assigned to the same broker, without this patch, their second replica is assigned to the same broker too.
        Neha Narkhede made changes -
        Labels p2
        Hide
        Guozhang Wang added a comment -

        Reviewed the patch, looks good to me.

        Just one question about the motivation: in normal settings where # brokers is larger than # replicas, even if the first replica of a partition is assigned to the same broker, their following replicas may not be assigned to the same broker as long as their partition id are not the same. Just from the example from the code: P1 of Topic1 and P5 of Topic2 could have their first replicas assigned to the same broker, while their following replicas are not assigned to the same brokers.

        Show
        Guozhang Wang added a comment - Reviewed the patch, looks good to me. Just one question about the motivation: in normal settings where # brokers is larger than # replicas, even if the first replica of a partition is assigned to the same broker, their following replicas may not be assigned to the same broker as long as their partition id are not the same. Just from the example from the code: P1 of Topic1 and P5 of Topic2 could have their first replicas assigned to the same broker, while their following replicas are not assigned to the same brokers.
        Jun Rao made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jun Rao made changes -
        Field Original Value New Value
        Attachment kafka-762_v1.patch [ 12569869 ]
        Hide
        Jun Rao added a comment -

        Attach patch v1. It addresses the problem by starting the second replica from a random interval after the first replica.

        Show
        Jun Rao added a comment - Attach patch v1. It addresses the problem by starting the second replica from a random interval after the first replica.
        Jun Rao created issue -

          People

          • Assignee:
            Jun Rao
            Reporter:
            Jun Rao
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development