[OAK-7379] Lucene Index: per-column selectivity, assume 5 unique entries

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.0, 1.10.0
• Component/s: lucene, query
• Epic Link: indexer resilience
• Assignee: Thomas Mueller
• Reporter: Thomas Mueller
• Created: 27/Mar/18 09:12
• Updated: 10/May/23 10:49
• Resolved: 11/Apr/18 11:32

Description

Currently, if a query has a property restriction of the form "property = x", and the property is indexed in a Lucene property index, the estimated cost is the index is the number of documents indexed for that property. This is a very conservative estimate, it means all documents have the same value. So the cost is relatively high for that index.

In almost all cases, there are many distinct values for a property. Rarely there are few values, or a skewed distribution where one value contains most documents. But in almost all cases there are more than 5 distinct values.

I think it makes sense to use 5 as the default value. It is still conservative (cost of the index is high), but much better than now.

Issue Links

• relates to: OAK-7300 Lucene Index: per-column selectivity to improve cost estimation