Kafka
  1. Kafka
  2. KAFKA-1548

Refactor the "replica_id" in requests

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:

      Description

      Today in many requests like fetch and offset we have a integer replica_id field, if the request is from a follower consumer it is the broker id from that follower replica, if it is from a regular consumer it could be one of the two values: "-1" for ordinary consumer, or "-2" for debugging consumer.

      Hence this replica_id field is used in two folds:

      1) Logging for trouble shooting in request logs, which can be helpful only when this is from a follower replica,

      2) Deciding if it is from the consumer or a replica to logically handle the request in different ways. For this purpose we do not really care about the actually id value.

      We probably would like to do the following improvements:

      1) Rename "replica_id" to sth. less confusing?

      2) Change the request.toString() function based on the replica_id, whether it is a positive integer (meaning from a broker replica fetcher) or -1/-2 (meaning from a regular consumer).

        Activity

        Hide
        Guozhang Wang added a comment -

        Hi Gwen,

        I kind of agree that for now maybe we do not need to change the name, with that then what we should probably do is:

        1. Add comments in RequestOrResponse explaining the semantics of replica id.
        2. Change the toString of Request with "Follower Replica Id", "Normal Consumer" and "Tooling Consumer" (this is for -2, the debugging consumer)

        Show
        Guozhang Wang added a comment - Hi Gwen, I kind of agree that for now maybe we do not need to change the name, with that then what we should probably do is: 1. Add comments in RequestOrResponse explaining the semantics of replica id. 2. Change the toString of Request with "Follower Replica Id", "Normal Consumer" and "Tooling Consumer" (this is for -2, the debugging consumer)
        Hide
        Gwen Shapira added a comment -

        I think the name replica_id is fine.

        For #2, I'm assuming the idea is to change the request printout in the log from:
        {replica_id=-1,max_wait_time=100,min_bytes=100000,topics=[{topic=test1,partitions=[

        {partition=0,fetch_offset=100,max_bytes=1000000}]},{topic=test2,partitions=[{partition=0,fetch_offset=200,max_bytes=1000000}]}]}

        to:
        {ordinary consumer,max_wait_time=100,min_bytes=100000,topics=[{topic=test1,partitions=[{partition=0,fetch_offset=100,max_bytes=1000000}

        ]},{topic=test2,partitions=[

        {partition=0,fetch_offset=200,max_bytes=1000000}

        ]}]}

        In cases where the replica_id is not from a follower.

        Did I get that right?

        Show
        Gwen Shapira added a comment - I think the name replica_id is fine. For #2, I'm assuming the idea is to change the request printout in the log from: {replica_id=-1,max_wait_time=100,min_bytes=100000,topics=[{topic=test1,partitions=[ {partition=0,fetch_offset=100,max_bytes=1000000}]},{topic=test2,partitions= [{partition=0,fetch_offset=200,max_bytes=1000000}] }]} to: {ordinary consumer,max_wait_time=100,min_bytes=100000,topics=[{topic=test1,partitions=[{partition=0,fetch_offset=100,max_bytes=1000000} ]},{topic=test2,partitions=[ {partition=0,fetch_offset=200,max_bytes=1000000} ]}]} In cases where the replica_id is not from a follower. Did I get that right?

          People

          • Assignee:
            Gwen Shapira
            Reporter:
            Guozhang Wang
            Reviewer:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development