Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1461

Replica fetcher thread does not implement any back-off behavior

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.1.1
    • 0.9.0.0
    • replication

    Description

      The current replica fetcher thread will retry in a tight loop if any error occurs during the fetch call. For example, we've seen cases where the fetch continuously throws a connection refused exception leading to several replica fetcher threads that spin in a pretty tight loop.

      To a much lesser degree this is also an issue in the consumer fetcher thread, although the fact that erroring partitions are removed so a leader can be re-discovered helps some.

      Attachments

        1. KAFKA-1461.patch
          19 kB
          Harsha
        2. KAFKA-1461.patch
          12 kB
          Harsha
        3. KAFKA-1461_2015-03-11_10:41:26.patch
          18 kB
          Harsha
        4. KAFKA-1461_2015-03-11_18:17:51.patch
          12 kB
          Harsha
        5. KAFKA-1461_2015-03-12_13:54:51.patch
          13 kB
          Harsha
        6. KAFKA-1461_2015-03-17_16:03:33.patch
          12 kB
          Harsha
        7. KAFKA-1461_2015-03-27_15:31:11.patch
          15 kB
          Harsha
        8. KAFKA-1461_2015-03-27_16:56:45.patch
          15 kB
          Harsha
        9. KAFKA-1461_2015-03-27_17:02:32.patch
          17 kB
          Harsha
        10. KAFKA-1461_2015-04-03_20:48:34.patch
          19 kB
          Harsha
        11. KAFKA-1461_2015-04-07_08:41:18.patch
          20 kB
          Harsha

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1629 Replica fetcher thread need to back off upon getting errors on partitions

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-2150 FetcherThread backoff need to grab lock before wait on condition.

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              sriharsha Harsha
              smeder Sam Meder
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: