Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
main (10.0)
-
None
-
None
Description
DocValues are used for the purpose of arbitrary-order value retrieval mainly in two places:
- ExportWriter (for the export handler)
- SolrDocumentFetcher (for useDocValuesAsStored)
ExportWriter holds a simple optimization to reuse docValues iterators where possible (i.e., where docId doesn't decrease), but there is no such optimization for SolrDocumentFetcher: for every possible docValues return field, for every doc, a new docValues iterator is pulled (and GC'd). This is especially pathological in the case of many sparse fields, because the hit is incurred regardless of whether the return fields are actually present in any returned docs.
This issue proposes unifying the docValues iterator caching for both these contexts, with the most immediate practical effect being improved latency of field retrieval for useDocValuesAsStored.
