Kafka
  1. Kafka
  2. KAFKA-930

Integrate preferred replica election logic into kafka

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.1
    • Component/s: None
    • Labels:
      None

      Description

      It seems useful to integrate the preferred replica election logic into kafka controller. A simple way to implement this would be to have a background thread that periodically finds the topic partitions that are not assigned to the preferred broker and initiate the move. We could come up with some heuristics to initiate the move only if the imbalance over a specific threshold in order to avoid rebalancing too aggressively. Making the software do this reduces operational cost.

      1. KAFKA-930_2013-11-19_17:37:29.patch
        9 kB
        Sriram Subramanian
      2. KAFKA-930_2013-11-19_17:38:49.patch
        9 kB
        Sriram Subramanian
      3. KAFKA-930_2013-11-21_09:42:11.patch
        9 kB
        Sriram Subramanian
      4. KAFKA-930_2013-12-09_22:51:57.patch
        96 kB
        Sriram Subramanian
      5. KAFKA-930_2013-12-20_11:13:01.patch
        119 kB
        Sriram Subramanian
      6. KAFKA-930_2013-12-20_11:22:36.patch
        117 kB
        Sriram Subramanian
      7. KAFKA-930_2014-01-27_13:28:51.patch
        5 kB
        Sriram Subramanian
      8. KAFKA-930_2014-02-24_01:59:46.patch
        6 kB
        Sriram Subramanian
      9. KAFKA-930.patch
        8 kB
        Sriram Subramanian

        Activity

        Sriram Subramanian created issue -
        Sriram Subramanian made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Sriram Subramanian made changes -
        Attachment KAFKA-930.patch [ 12614757 ]
        Hide
        Sriram Subramanian added a comment -

        Created reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Created reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-11-19_17:37:29.patch [ 12614763 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-11-19_17:38:49.patch [ 12614764 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-11-21_09:42:11.patch [ 12615155 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-12-09_22:51:57.patch [ 12618002 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-12-20_11:13:01.patch [ 12619892 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2013-12-20_11:22:36.patch [ 12619894 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Hide
        Sriram Subramanian added a comment -

        checked in to trunk

        Show
        Sriram Subramanian added a comment - checked in to trunk
        Sriram Subramanian made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Sriram Subramanian made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2014-01-27_13:28:51.patch [ 12625445 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Joe Stein made changes -
        Fix Version/s 0.8.1 [ 12322960 ]
        Fix Version/s 0.9.0 [ 12323928 ]
        Sriram Subramanian made changes -
        Attachment KAFKA-930_2014-02-24_01:59:46.patch [ 12630639 ]
        Hide
        Sriram Subramanian added a comment -

        Updated reviewboard https://reviews.apache.org/r/15711/
        against branch origin/trunk

        Show
        Sriram Subramanian added a comment - Updated reviewboard https://reviews.apache.org/r/15711/ against branch origin/trunk
        Hide
        Sriram Subramanian added a comment -

        final changes pushed to 0.8.1 and trunk

        Show
        Sriram Subramanian added a comment - final changes pushed to 0.8.1 and trunk

          People

          • Assignee:
            Sriram Subramanian
            Reporter:
            Sriram Subramanian
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development