Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
New
Description
Lucene 8.5.1 includes a change to always compress BinaryDocValues. This caused (~30%) reduction in our red-line QPS (throughput).
We think users should be given some way to opt-in for this compression feature instead of always being enabled which can have a substantial query time cost as we saw during our upgrade. mikemccand suggested one possible approach by introducing a mode in Lucene80DocValuesFormat (COMPRESSED and UNCOMPRESSED) and allowing users to create a custom Codec subclassing the default Codec and pick the format they want.
Idea is similar to Lucene50StoredFieldsFormat which has two modes, Mode.BEST_SPEED and Mode.BEST_COMPRESSION.
Here's related issues for adding benchmark covering BINARY doc values query-time performance - https://github.com/mikemccand/luceneutil/issues/61
Attachments
Attachments
Issue Links
- is caused by
-
LUCENE-9211 Adding compression to BinaryDocValues storage
- Resolved
- links to