[LUCENE-2431] A Real-Time Split Policy For Searching In Real-Time

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.1
• Component/s: core/index
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Karthick Sankarachary
• Created: 02/May/10 16:31
• Updated: 28/Aug/22 12:25

Description

(Please note that the feature described in LUCENE-2425 is required reading for this issue.)

The real-time policy makes it possible to search and read from a (Lucene) split index in a real-time manner. Specifically, changes made to the split writer will immediately be reflected in all readers that were obtained from it.

Note that, while the readers based on this policy are real-time, in order to truly achieve real-time searching, one must use a real time searcher, which in turn builds on the index searcher (see attached test cases for more details).

Now, we describe how this policy overcomes the near-real time limitation of current indices. For every reader, it creates a split in the form of a RAM-based directory, which serves to track changes made to the split index in between the time that and the next reader was obtained. When a new reader is obtained, the corresponding split is dynamically added to that reader, and all the readers that came before it. Note that a reader does not need to (nor does it) see the splits that came before it, since the snapshot of the super-reader that it obtains is current at the time of its inception.

Issue Links

• requires: LUCENE-2425 An Anti-Merging Multi-Directory Indexing Framework