[LUCENE-9850] Explore PFOR for Doc ID delta encoding (instead of FOR) - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Task
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 9.0
Fix Version/s: 9.0
Component/s: core/codecs
Labels:
None

Lucene Fields:

New

Description

It'd be interesting to explore using PFOR instead of FOR for doc ID encoding. Right now PFOR is used for positions, frequencies and payloads, but FOR is used for doc ID deltas. From a recent conversation on the dev mailing list, it sounds like this decision was made based on the optimization possible when expanding the deltas.

I'd be interesting in measuring the index size reduction possible with switching to PFOR compared to the performance reduction we might see by no longer being able to apply the deltas in as optimal a way.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

for.png
02/Apr/21 17:59
207 kB
Greg Miller
pfor.png
02/Apr/21 18:00
250 kB
Greg Miller
apply_exceptions.png
02/Apr/21 18:05
146 kB
Greg Miller
bulk_read_1.png
02/Apr/21 23:48
234 kB
Greg Miller
bulk_read_2.png
02/Apr/21 23:49
154 kB
Greg Miller

Issue Links

links to

GitHub Pull Request #69

Activity

People

Assignee:: Unassigned

Reporter:: Greg Miller

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 17/Mar/21 22:23

Updated:: 28/Aug/22 16:18

Resolved:: 14/Apr/21 12:42

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

6h 40m