[SOLR-5674] The rows improvement for QueryComponent

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 4.3.1, 4.5.1, 4.6
• Component/s: search
• Labels: QueryComponent

Environment

JVM7

• Assignee: Unassigned
• Reporter: Raintung Li
• Created: 29/Jan/14 07:32
• Updated: 11/Jan/18 18:07
• Resolved: 11/Jan/18 18:07

Description

For solr Rows issues:
1. Solr don't provide get full results API, usually customer will set the rows is Integer.maxvalue try to get the full results that cause the other issue. OOM issue in solr :SOLR-5661(https://issues.apache.org/jira/browse/SOLR-5661)
How about open the API for rows=-1? That means return full results. Sometimes the result count will every biggest that will cause the heap OOM, but usually we can suggest the customer to make sure the result really small that can call this API. Actually we don't want to make the second call to get full results. For one is call API get total number, for two get the result set rows into total number.

2. A litter improve, because every shard node return results has been ordered. Add first shard list into the PriorityQueue that don't need compare again only filter the same unique id.

3. Create the PriorityQueue after check all shard return sizes, that can avoid the unnecessary memory cost especially biggest rows.