[LUCENE-7806] Explore delta of delta encoding

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: compression docValues
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Otis Gospodnetic
• Created: 25/Apr/17 17:41
• Updated: 28/Aug/22 15:14

Description

From http://search-lucene.com/m/Lucene/l6pAi1YEfXhuGGIl1?subj=Re+Delta+of+delta+encoding

delta of delta encoding is one of the Facebook Gorilla tricks that allows it to compress 16 bytes into 1.37 bytes on average – see section 4.1 that describes it – http://www.vldb.org/pvldb/vol8/p1816-teller.pdf

This seems to be aimed at both time fields and numerical values.

https://github.com/burmanm/gorilla-tsc is a recent Java impl

CC jpountz

Issue Links

• depends upon: LUCENE-10033 Encode doc values in smaller blocks of values, like postings