[LUCENE-1990] Add unsigned packed int impls in oal.util

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0-ALPHA
• Fix Version/s: 3.5, 4.0-ALPHA
• Component/s: core/index
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Michael McCandless
• Created: 18/Oct/09 12:53
• Updated: 28/Aug/22 12:11
• Resolved: 06/Apr/10 20:58

Description

There are various places in Lucene that could take advantage of an
efficient packed unsigned int/long impl. EG the terms dict index in
the standard codec in LUCENE-1458 could subsantially reduce it's RAM
usage. FieldCache.StringIndex could as well. And I think "load into
RAM" codecs like the one in TestExternalCodecs could use this too.

I'm picturing something very basic like:

interface PackedUnsignedLongs  {
  long get(long index);
  void set(long index, long value);
}

Plus maybe an iterator for getting and maybe also for setting. If it
helps, most of the usages of this inside Lucene will be "write once"
so eg the set could make that an assumption/requirement.

And a factory somewhere:

  PackedUnsignedLongs create(int count, long maxValue);

I think we should simply autogen the code (we can start from the
autogen code in LUCENE-1410), or, if there is an good existing impl
that has a compatible license that'd be great.

I don't have time near-term to do this... so if anyone has the itch,
please jump!

Issue Links

• blocks: LUCENE-2186 First cut at column-stride fields (index values storage)
• blocks: LUCENE-2141 Make String and StringIndex in field cache more RAM efficient
• is related to: LUCENE-2633 PackedInts does not support structures above 256MB