Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1461

Replica fetcher thread does not implement any back-off behavior

          Details

          • Type: Improvement
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 0.8.1.1
          • Fix Version/s: 0.9.0.0
          • Component/s: replication
          • Labels:

            Description

            The current replica fetcher thread will retry in a tight loop if any error occurs during the fetch call. For example, we've seen cases where the fetch continuously throws a connection refused exception leading to several replica fetcher threads that spin in a pretty tight loop.

            To a much lesser degree this is also an issue in the consumer fetcher thread, although the fact that erroring partitions are removed so a leader can be re-discovered helps some.

              Attachments

              1. KAFKA-1461_2015-03-11_10:41:26.patch
                18 kB
                Sriharsha Chintalapani
              2. KAFKA-1461_2015-03-11_18:17:51.patch
                12 kB
                Sriharsha Chintalapani
              3. KAFKA-1461_2015-03-12_13:54:51.patch
                13 kB
                Sriharsha Chintalapani
              4. KAFKA-1461_2015-03-17_16:03:33.patch
                12 kB
                Sriharsha Chintalapani
              5. KAFKA-1461_2015-03-27_15:31:11.patch
                15 kB
                Sriharsha Chintalapani
              6. KAFKA-1461_2015-03-27_16:56:45.patch
                15 kB
                Sriharsha Chintalapani
              7. KAFKA-1461_2015-03-27_17:02:32.patch
                17 kB
                Sriharsha Chintalapani
              8. KAFKA-1461_2015-04-03_20:48:34.patch
                19 kB
                Sriharsha Chintalapani
              9. KAFKA-1461_2015-04-07_08:41:18.patch
                20 kB
                Sriharsha Chintalapani
              10. KAFKA-1461.patch
                12 kB
                Sriharsha Chintalapani
              11. KAFKA-1461.patch
                19 kB
                Sriharsha Chintalapani

                Issue Links

                is duplicated by

                Bug - A problem which impairs or prevents the functions of the product. KAFKA-1629 Replica fetcher thread need to back off upon getting errors on partitions

                • Major - Major loss of function.
                • Resolved
                relates to

                Bug - A problem which impairs or prevents the functions of the product. KAFKA-2150 FetcherThread backoff need to grab lock before wait on condition.

                • Major - Major loss of function.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      sriharsha Sriharsha Chintalapani
                      Reporter:
                      smeder Sam Meder
                    • Votes:
                      2 Vote for this issue
                      Watchers:
                      6 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: