[LUCENE-5688] NumericDocValues fields with sparse data can be compressed better

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Varun Thacker
• Created: 20/May/14 13:22
• Updated: 28/Aug/22 14:07
• Resolved: 06/Nov/15 15:35

Description

I ran into this problem where I had a dynamic field in Solr and indexed data into lots of fields. For each field only a few documents had actual values and the remaining documents the default value ( 0 ) got indexed. Now when I merge segments, the index size jumps up.

For example I have 10 segments - Each with 1 DV field. When I merge segments into 1 that segment will contain all 10 DV fields with lots if 0s.

This was the motivation behind trying to come up with a compression for a use case like this.

Issue Links

• is superceded by: LUCENE-6863 Store sparse doc values more efficiently