Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9290

TCP-connections in CLOSE_WAIT spike during heavy indexing and do not decrease

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 5.3.2, 5.4.1, 5.5.1, 5.5.2, 6.0, 6.0.1, 6.1
    • 5.5.3, 6.2
    • None
    • None

    Description

      Heavy indexing on Solr with SSL leads to a lot of connections in CLOSE_WAIT state.

      At my workplace, we have seen this issue only with 5.5.1 and could not reproduce it with 5.4.1 but from my conversation with Shalin, he knows of users with 5.3.1 running into this issue too.

      Here's an excerpt from the email shaie sent to the mailing list (about what we see:

      1) It consistently reproduces on 5.5.1, but does not reproduce on 5.4.1
      2) It does not reproduce when SSL is disabled
      3) Restarting the Solr process (sometimes both need to be restarted), the
      count drops to 0, but if indexing continues, they climb up again

      When it does happen, Solr seems stuck. The leader cannot talk to the
      replica, or vice versa, the replica is usually put in DOWN state and
      there's no way to fix it besides restarting the JVM.

      Here's the mail thread: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201607.mbox/%3C46cc66220a8143dc903fa34e792059c4@vp-exc01.dips.local%3E

      Creating this issue so we could track this and have more people comment on what they see.

      Attachments

        1. index.sh
          0.8 kB
          Chris M. Hostetter
        2. setup-solr.sh
          1 kB
          Chris M. Hostetter
        3. setup-solr.sh
          0.6 kB
          Shalin Shekhar Mangar
        4. SOLR-9290.patch
          19 kB
          Shalin Shekhar Mangar
        5. SOLR-9290.patch
          17 kB
          Shalin Shekhar Mangar
        6. SOLR-9290.patch
          17 kB
          Shalin Shekhar Mangar
        7. SOLR-9290-debug.patch
          17 kB
          Shalin Shekhar Mangar
        8. SOLR-9290-debug.patch
          15 kB
          Shalin Shekhar Mangar

        Activity

          People

            shalin Shalin Shekhar Mangar
            anshum Anshum Gupta
            Votes:
            0 Vote for this issue
            Watchers:
            21 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: