[#LUCENE-328] Some utilities for a compact sparse filter

Lucene - Java

Some utilities for a compact sparse filter

Created: 04/Jan/05 12:38 AM Updated: 28/Jun/06 01:38 AM

Return to search

Component/s:

Affects Version/s:

CVS Nightly - Specify date in submission

Fix Version/s:

None

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
AndDocNrSkipper.java	2005-06-21 07:52 PM	Mark Harwood	2 kB
AndDocNrSkipper.java	2005-06-21 06:21 PM	Mark Harwood	2 kB
BitSetSortedIntList.java	2005-06-21 06:20 PM	Mark Harwood	1 kB
DocNrSkipper.java	2005-02-09 04:43 AM	Paul Elschot	1 kB
DocNrSkipper.java	2005-01-17 02:00 AM	Paul Elschot	1 kB
IntArraySortedIntList.java	2005-11-22 07:07 AM	Mark Harwood	3 kB
IntArraySortedIntList.java	2005-06-21 06:19 PM	Mark Harwood	3 kB
OrDocNrSkipper.java	2005-06-21 07:53 PM	Mark Harwood	2 kB
OrDocNrSkipper.java	2005-06-21 06:20 PM	Mark Harwood	2 kB
SkipFilter1.patch	2006-05-15 11:17 PM	Paul Elschot	4 kB
SortedVIntList.java	2005-02-09 04:44 AM	Paul Elschot	4 kB
SortedVIntList.java	2005-01-17 02:04 AM	Paul Elschot	4 kB
SortedVIntList.java	2005-01-04 12:40 AM	Paul Elschot	4 kB
TestDocNrSkippers.java	2005-06-21 07:53 PM	Mark Harwood	6 kB
TestDocNrSkippers.java	2005-06-21 06:21 PM	Mark Harwood	6 kB
TestSortedVIntList.java	2005-02-09 04:46 AM	Paul Elschot	4 kB
TestSortedVIntList.java	2005-01-17 02:06 AM	Paul Elschot	4 kB
TestSortedVIntList.java	2005-01-06 03:21 AM	Paul Elschot	4 kB

Environment:

« Hide

Operating System: other
Platform: Other

Issue Links:

Reference

This issue is related to:

~~LUCENE-584~~ Decouple Filter from BitSet

Bugzilla Id:	32921
Resolution Date:	28/Jun/06 01:38 AM

Description

« Hide

Two files are attached that might form the basis for an alternative
filter implementation that is more memory efficient than one bit
per doc when less than about 1/8 of the docs pass through the filter.

The document numbers are stored in RAM as VInt's from the Lucene index
format. These VInt's encode the difference between two successive
document numbers, much like a PositionDelta in the Positions:
http://jakarta.apache.org/lucene/docs/fileformats.html

The getByteSize() method can be used to verify the compression
once a SortedVIntList is constructed.
The precise conditions under which this is more memory efficient than
one bit per document are not easy to specify in advance.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Jeff Turner made changes - 03/Sep/05 03:28 PM

Field	Original Value	New Value
issue.field.bugzillaimportkey	32921	12314478

Mark Harwood made changes - 22/Nov/05 07:07 AM

Attachment

IntArraySortedIntList.java [ 12320844 ]

Paul Elschot made changes - 03/Jan/06 07:14 AM

Attachment

intIterator.java [ 12312473 ]

Paul Elschot made changes - 15/May/06 11:17 PM

Attachment

SkipFilter1.patch [ 12331912 ]

Paul Elschot made changes - 27/Jun/06 05:09 AM

Link

This issue is related to ~~LUCENE-584~~ [ ~~LUCENE-584~~ ]

Otis Gospodnetic made changes - 28/Jun/06 01:38 AM

Resolution		Duplicate [ 3 ]
Assignee	Lucene Developers [ java-dev@lucene.apache.org ]
Status	Open [ 1 ]	Resolved [ 5 ]