[SOLR-4903] Solr sends all doc ids to all shards in the query counting facets

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.4, 4.3, 4.3.1
• Component/s: search
• Assignee: Unassigned
• Reporter: Dmitry Kan
• Created: 06/Jun/13 06:40
• Updated: 11/Apr/21 07:44
• Resolved: 11/Apr/21 07:44

Description

Setup: front end solr and shards.

Summary: solr frontend sends all doc ids received from QueryComponent to all shards which causes POST request buffer size overflow.

Symptoms:

The query is: http://pastebin.com/0DndK1Cs
I have omitted the shards parameter.

The router log: http://pastebin.com/FTVH1WF3
Notice the port of a shard, that is affected. That port changes all the time, even for the same request
The log entry is prepended with lines:

SEVERE: org.apache.solr.common.SolrException: Internal Server Error

Internal Server Error

(they are not in the pastebin link)

The shard log: http://pastebin.com/exwCx3LX

Suggestion: change the data structure in FacetComponent to send only doc ids that belong to a shard and not a concatenation of all doc ids.

Why is this important: for scaling. Adding more shards will result in overflowing the POST request buffer at some point anyway.

Issue Links

• is related to: SOLR-4904 Send internal doc ids and index version in distributed faceting to make queries more compact