[SOLR-8096] Major faceting performance regressions

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 5.0, 5.1, 5.2, 5.3, 6.0
• Assignee: Unassigned
• Reporter: Yonik Seeley
• Created: 25/Sep/15 03:15
• Updated: 08/Sep/19 18:17

Description

Use of the highly optimized faceting that Solr had for multi-valued fields over relatively static indexes was removed as part of LUCENE-5666, causing severe performance regressions.

Here are some quick benchmarks to gauge the damage, on a 5M document index, with each field having between 0 and 5 values per document. Higher numbers represent worse 5x performance.

Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time

...................................	Percent of index being faceted

num_unique_values	10%	50%	90%
10	351.17%	1587.08%	3057.28%
100	158.10%	203.61%	1421.93%
1000	143.78%	168.01%	1325.87%
10000	137.98%	175.31%	1233.97%
100000	142.98%	159.42%	1252.45%
1000000	255.15%	165.17%	1236.75%

For example, a field with 1000 unique values in the whole index, faceting with 5x took 143% of the 4x time, when ~10% of the docs in the index were faceted.

One user who brought the performance problem to our attention: http://markmail.org/message/ekmqh4ocbkwxv3we
"faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3)

The disabling of the UnInvertedField algorithm was previously discovered in SOLR-7190, but we didn't know just how bad the problem was at that time.

edit: removed "secret" adverb by request

Issue Links

• is broken by: LUCENE-5666 Add UninvertingReader
• is part of: SOLR-8466 Add support for UnInvertedField based faceting to FacetComponent
• is related to: SOLR-7730 speed-up faceting on doc values fields
• relates to: SOLR-9176 Legacy Faceting Term Enum Method Regression
• links to: problem resolution