[LUCENE-6668] Optimize SortedSet/SortedNumeric storage for the few unique sets use-case

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 5.3
• Lucene Fields: New
• Assignee: Adrien Grand
• Reporter: Adrien Grand
• Created: 09/Jul/15 12:18
• Updated: 28/Aug/22 14:38
• Resolved: 21/Jul/15 08:47

Description

Robert suggested this idea: if there are few unique sets of values, we could build a lookup table and then map each doc to an ord in this table, just like we already do for table compression for numerics.

I think this is especially compelling given that SortedSet/SortedNumeric are our two only doc values types that use O(maxDoc) memory because of the offsets map. When this new strategy is used, memory usage could be bounded to a constant.