[SOLR-14457] SolrClient leaks connections on compressed responses if the response is malformed

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.7.2
• Fix Version/s: 8.10
• Component/s: SolrJ

Environment

Solr version: 7.7.2

Solr cloud enabled

Cluster topology: 6 nodes, 1 single collection, 10 shards and 3 replicas. 1 HTTP LB using

Round Robin over all nodes

All cluster nodes have gzip enabled for all paths, all HTTP verbs and all MIME types.

Solr client: HttpSolrClient targeting the HTTP LB

• Assignee: Houston Putman
• Reporter: Samuel García Martínez
• Created: 02/May/20 10:29
• Updated: 28/Sep/21 20:22
• Resolved: 04/Sep/21 16:47

Description

Summary

When the SolrJ receives a malformed response Entity, for example like the one described in SOLR-14456, the client leaks the connection forever as it's never released back to the pool.

Problem description

HttpSolrClient should have compression enabled, so it uses the compression interceptors.

When the request is marked with "Content-Encoding: gzip" but for whatever reason the response is not in GZIP format, when  HttpSolrClient tries to close the connection using Utils.consumeFully(), it tries to create the GzipInputStream failing and throwing an Exception. The exception thrown makes it impossible to access the underlying InputStream to be closed, therefore the connection is leaked.

Despite that the content in the response should honour the headers specified for it, SolrJ should be reliable enough to prevent the connection leak when this scenario happens. On top of the bug itself, not being able to set a timeout while waiting for a connection to be available, makes any application unresponsive as it will run out of threads eventually.

Issue Links

• is related to: SOLR-14456 Compressed requests fail in SolrCloud when the request is routed internally by the serving solr node
• links to: GitHub Pull Request #283