[LUCENE-2793] Directory createOutput and openInput should take an IOContext

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0-ALPHA, IOContext branch
• Component/s: core/store
• Labels: gsoc2011 lucene-gsoc-11 mentor
• Lucene Fields: New, Patch Available
• Assignee: Varun Thacker
• Reporter: Michael McCandless
• Created: 03/Dec/10 15:47
• Updated: 28/Aug/22 12:37
• Resolved: 08/Jul/11 08:53

Description

Today for merging we pass down a larger readBufferSize than for searching because we get better performance.

I think we should generalize this to a class (IOContext), which would hold the buffer size, but then could hold other flags like DIRECT (bypass OS's buffer cache), SEQUENTIAL, etc.

Then, we can make the DirectIOLinuxDirectory fully usable because we would only use DIRECT/SEQUENTIAL during merging.

This will require fixing how IW pools readers, so that a reader opened for merging is not then used for searching, and vice/versa. Really, it's only all the open file handles that need to be different – we could in theory share del docs, norms, etc, if that were somehow possible.

Issue Links

• blocks: LUCENE-3178 Native MMapDir
• blocks: LUCENE-3203 Rate-limit IO used by merging
• blocks: LUCENE-2795 Genericize DirectIOLinuxDir -> UnixDir
• blocks: LUCENE-3092 NRTCachingDirectory, to buffer small segments in a RAMDir
• is depended upon by: LUCENE-3292 IOContext should be part of the SegmentReader cache key
• is depended upon by: LUCENE-3293 Use IOContext.READONCE in VarGapTermsIndexReader to load FST