[LUCENE-4936] docvalues date compression

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.4
• Component/s: core/index
• Lucene Fields: New
• Assignee: Adrien Grand
• Reporter: Robert Muir
• Created: 16/Apr/13 12:42
• Updated: 28/Aug/22 13:44
• Resolved: 29/Apr/13 18:07

Description

DocValues fields can be very wasteful if you are storing dates (like solr's TrieDateField does if you enable docvalues) and don't actually need all the precision: e.g. "date-only" fields like date of birth with no time component, time fields without milliseconds precision, and so on.

Ideally we'd compute GCD of all the values to save space (numberOfTrailingZeros is not really enough here), but i think we should at least look for values like 86400000, 3600000, and 1000 to be practical.