[LUCENE-2432] A Caching Split Policy For Real-Time Index Caching

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.1
• Component/s: core/index, modules/other
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Karthick Sankarachary
• Created: 02/May/10 16:48
• Updated: 28/Aug/22 12:25

Description

(Please note that the feature described in LUCENE-2425 is required reading for this issue.)

he caching split policy builds on the real-time split policy, providing a caching mechanism that serves to buffer writes to the split index.

As one might expect, this policy delays writes to the index, holding them in the RAM-based splits (created by the real-time policy) instead. When a flush is forced (typically through a commit, but in general, through a split rule), then all un-flushed changes (splits) are written to the super-directory.

There are a couple of reasons why one may wish to use the caching split policy opposed to a RAMDirectory. The latter is primarily used to "load a disk-based index into memory", and suffers from a limitation: "the resulting RAMDirectory instance is fully independent from the original Directory (it is a complete copy). Any subsequent changes to the original Directory will not be visible in the RAMDirectory instance". Furthermore, the RAMDirectory gives little control over when changes are flushed to the disk. The caching split policy attempts to address those limitations by allowing (a) changes to the original directory (which we refer to as the super-directory) are in fact visible to the reader and (b) the user to define exactly when to flush writes to the underlying directory.

Issue Links

• requires: LUCENE-2425 An Anti-Merging Multi-Directory Indexing Framework