Solr
  1. Solr
  2. SOLR-6931

We should do a limited retry when using HttpClient.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.10.4, 5.0, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      This is likely the issue you are looking for if you are looking to solve random SocketException "connection reset" issues.

      1. SOLR-6931.patch
        8 kB
        Mark Miller
      2. SOLR-6931.patch
        7 kB
        Mark Miller
      3. SOLR-6931.patch
        4 kB
        Mark Miller

        Issue Links

          Activity

          Hide
          Mark Miller added a comment -

          See SOLR-4509. HttpClient uses a stale connection check to avoid using a bad pooled connection. This check has a race, and we can use a bad connection sometimes. In most of these cases, it is actually safe for us to retry. We can't use the default retry handler because it attempts to detect idempotent updates and Solr allows update type requests via GET requests. If we turn off the idempotent detection, the retry is safe and we can avoid some very problematic problems like 'connection reset' exceptions. On a heavy working SolrCloud cluster, even a rare response like this from a replica can cause a recovery and heavy cluster disruption.

          Show
          Mark Miller added a comment - See SOLR-4509 . HttpClient uses a stale connection check to avoid using a bad pooled connection. This check has a race, and we can use a bad connection sometimes. In most of these cases, it is actually safe for us to retry. We can't use the default retry handler because it attempts to detect idempotent updates and Solr allows update type requests via GET requests. If we turn off the idempotent detection, the retry is safe and we can avoid some very problematic problems like 'connection reset' exceptions. On a heavy working SolrCloud cluster, even a rare response like this from a replica can cause a recovery and heavy cluster disruption.
          Hide
          Mark Miller added a comment -

          I'll commit this one soon so it is in for 5.0.

          Show
          Mark Miller added a comment - I'll commit this one soon so it is in for 5.0.
          Hide
          Mark Miller added a comment -

          Improved patch. By default, search side requests will continue to avoid a retry by default. With replicas, you might prefer another is tried instead. You can configure retries on the search side though.

          Show
          Mark Miller added a comment - Improved patch. By default, search side requests will continue to avoid a retry by default. With replicas, you might prefer another is tried instead. You can configure retries on the search side though.
          Hide
          ASF subversion and git services added a comment -

          Commit 1651767 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1651767 ]

          SOLR-6931: We should do a limited retry when using HttpClient.

          Show
          ASF subversion and git services added a comment - Commit 1651767 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1651767 ] SOLR-6931 : We should do a limited retry when using HttpClient.
          Hide
          ASF subversion and git services added a comment -

          Commit 1651770 from Mark Miller in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1651770 ]

          SOLR-6931: We should do a limited retry when using HttpClient.

          Show
          ASF subversion and git services added a comment - Commit 1651770 from Mark Miller in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1651770 ] SOLR-6931 : We should do a limited retry when using HttpClient.
          Hide
          ASF subversion and git services added a comment -

          Commit 1651775 from Mark Miller in branch 'dev/branches/lucene_solr_5_0'
          [ https://svn.apache.org/r1651775 ]

          SOLR-6931: We should do a limited retry when using HttpClient.

          Show
          ASF subversion and git services added a comment - Commit 1651775 from Mark Miller in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1651775 ] SOLR-6931 : We should do a limited retry when using HttpClient.
          Hide
          Lindsay Martin added a comment -

          Is it possible to apply this to the 4.10.x branch? This would help us out with https://issues.apache.org/jira/browse/SOLR-6983

          Show
          Lindsay Martin added a comment - Is it possible to apply this to the 4.10.x branch? This would help us out with https://issues.apache.org/jira/browse/SOLR-6983
          Hide
          ASF subversion and git services added a comment -

          Commit 1652408 from Mark Miller in branch 'dev/branches/lucene_solr_4_10'
          [ https://svn.apache.org/r1652408 ]

          SOLR-6931: We should do a limited retry when using HttpClient.

          Show
          ASF subversion and git services added a comment - Commit 1652408 from Mark Miller in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1652408 ] SOLR-6931 : We should do a limited retry when using HttpClient.
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development