Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1461

Replica fetcher thread does not implement any back-off behavior

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1.1
    • Fix Version/s: 0.9.0.0
    • Component/s: replication
    • Labels:

      Description

      The current replica fetcher thread will retry in a tight loop if any error occurs during the fetch call. For example, we've seen cases where the fetch continuously throws a connection refused exception leading to several replica fetcher threads that spin in a pretty tight loop.

      To a much lesser degree this is also an issue in the consumer fetcher thread, although the fact that erroring partitions are removed so a leader can be re-discovered helps some.

        Attachments

        1. KAFKA-1461.patch
          19 kB
          Sriharsha Chintalapani
        2. KAFKA-1461.patch
          12 kB
          Sriharsha Chintalapani
        3. KAFKA-1461_2015-04-07_08:41:18.patch
          20 kB
          Sriharsha Chintalapani
        4. KAFKA-1461_2015-04-03_20:48:34.patch
          19 kB
          Sriharsha Chintalapani
        5. KAFKA-1461_2015-03-27_17:02:32.patch
          17 kB
          Sriharsha Chintalapani
        6. KAFKA-1461_2015-03-27_16:56:45.patch
          15 kB
          Sriharsha Chintalapani
        7. KAFKA-1461_2015-03-27_15:31:11.patch
          15 kB
          Sriharsha Chintalapani
        8. KAFKA-1461_2015-03-17_16:03:33.patch
          12 kB
          Sriharsha Chintalapani
        9. KAFKA-1461_2015-03-12_13:54:51.patch
          13 kB
          Sriharsha Chintalapani
        10. KAFKA-1461_2015-03-11_18:17:51.patch
          12 kB
          Sriharsha Chintalapani
        11. KAFKA-1461_2015-03-11_10:41:26.patch
          18 kB
          Sriharsha Chintalapani

          Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1629 Replica fetcher thread need to back off upon getting errors on partitions

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-2150 FetcherThread backoff need to grab lock before wait on condition.

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                sriharsha Sriharsha Chintalapani
                Reporter:
                smeder Sam Meder
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: