[LUCENE-6617] Reduce FST's ram usage

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.3, 6.0
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 26/Jun/15 16:25
• Updated: 28/Aug/22 14:37
• Resolved: 30/Jun/15 10:07

Description

Spinoff from LUCENE-6199, pulling out just the FST RAM reduction changes.

The FST data structure tries to be a RAM efficient representation of a sorted map, but there are a few things I think we can do to trim it even more:

• Don't store arc and node count: this is available from the Builder if you really want to do something with it.

• Don't use the "paged" byte store unless the FST is huge; just use a single byte[]

• Some members like lastFrozenNode, reusedBytesPerArc, allowArrayArcs are only used during building, so we should move them to the Builder

• We don't need to cache NO_OUTPUT: we can ask the Outputs impl for it