org.apache.lucene.index
Class IdleTimeoutRefreshingIndexReader

java.lang.Object
  |
  +--org.apache.lucene.index.IndexReader
        |
        +--org.apache.lucene.index.IdleTimeoutRefreshingIndexReader

public class IdleTimeoutRefreshingIndexReader

extends IndexReader

This index reader delegates most tasks to the normal IndexReader implementation. It adds support for refreshing a stale index (one that has been modified since it was opened) without requiring the consumer of the reader to be aware of all the details. It also supplies a mechanism to close the underlying reader when it has gone idle and re-open it when it is used again.

There are four configurable properties:

#minimumWaitToClose (static member)

The minimum allowed value of the waitToClose property.

#waitToClose

The time in milliseconds to wait before closing an index that has timed out or been reopened for a refresh. This should allow enough time for any current consumers to wrap up their use of the reader delegate. This IdleTimeoutRefreshingIndexReader is still available for reads and will have a new delegate open to service requests.

#timeout

The time in milliseconds to wait before closing an index due to inactivity.

#refreshInterval

The time in milliseconds to wait before examining an index to determine if it needs to be refreshed.

The minimumWaitToClose property can be configured using setMinimumWaitToClose(long). Each of the other properties can be configured on a per instance basis using the property setters setWaitToClose(long), setTimeout(long) and setRefreshInterval(long).

Defaults can be set on the command line by specifying long values as system properties using the keys PROP_WAIT_TO_CLOSE, PROP_TIMEOUT and PROP_REFRESH_INTERVAL. If those defaults are not set or don't parse as longs according to Long.getLong(String), the default is to never timeout and never refresh (behavior analogous to a normal IndexReader with a performance penalty due to the delegation model).

The defaults can also be manipulated at runtime using the static setters setDefaultWaitToClose(long), setDefaultTimeout(long) and setDefaultRefreshInterval(long). Runtime changes to the defaults will not affect already instantiated instances, use the instance level setters instead.

The waitToClose property has a minimum setting determined by MINIMUM_WAIT_TO_CLOSE. It is important that the closing reader have enough time to complete the servicing of any existing requests before being closed. This strategy was chosen in lieu of completely synchronizing all requests to the index reader. If the reader is closed to soon the consumers will get an IOException since they will be attempting to read from closed files.

There were three methods of IndexReader that did not throw IOException. These were wrapped to throw RuntimeException. See isDeleted(int), maxDoc() and numDocs(). I'm not sure this was a good strategy, but I wasn't sure what else to choose.

I'm not sure at the time of this writing whether this implementation works with a RAMDirectory implementation.

Version:

$Id$

Author:

Eric Isakson

Field Summary

static long	DEFAULT_MINIMUM_WAIT_TO_CLOSE The default minimum value of the waitToClose property used if no default was specified and the system property is not set.
static long	DEFAULT_REFRESH_INTERVAL The default value of the refreshInterval property.
static long	DEFAULT_TIMEOUT The default value of the timeout property.
static long	DEFAULT_WAIT_TO_CLOSE The default value of the waitToClose property.
static String	PROP_MINIMUM_WAIT_TO_CLOSE The name of the minimumWaitToClose property.
static String	PROP_REFRESH_INTERVAL The name of the refreshInterval property.
static String	PROP_TIMEOUT The name of the timeout property.
static String	PROP_WAIT_TO_CLOSE The name of the waitToClose property.

Constructor Summary

protected

IdleTimeoutRefreshingIndexReader(Directory directory, long timeout, long refreshInterval, long waitToClose) Constructs an IdleTimeoutRefreshingIndexReader.

Method Summary

int	docFreq(Term arg0) Returns the number of documents containing the term t.
protected void	doClose() Closes up any monitoring threads and the reader delegate if they exist.
Document	document(int arg0) Returns the stored fields of the nth Document in this index.
protected void	doDelete(int arg0)
static long	getDefaultRefreshInterval() Gets the default value for the refreshInterval property.
static long	getDefaultTimeout() Gets the default value for the timeout property.
static long	getDefaultWaitToClose() Gets the default value for the waitToClose property.
Collection	getFieldNames() Returns a list of all unique field names that exist in the index pointed to by this IndexReader.
Collection	getFieldNames(boolean arg0) Returns a list of all unique field names that exist in the index pointed to by this IndexReader.
long	getLastAccessed() Gets the last accessed time for this reader.
static long	getMinimumWaitToClose() Gets the minimum value for the waitToClose property.
long	getModifiedTimeOnOpen() Gets the time this index reported as its last modified time when it was opened.
long	getRefreshInterval() Gets the refresh interval used to check for changes to the index being read.
long	getTimeout() Gets the idle timeout used to close inactive readers.
long	getWaitToClose() Gets the wait time used to give consumers of this reader time to finish up before the underlying files are closed.
boolean	isDeleted(int arg0) Returns true if document n has been deleted
boolean	isIdle() Checks if the reader has been idle longer than the idle timeout.
boolean	isStale() Checks if the index has changed since it was opened.
int	maxDoc() Returns one greater than the largest possible document number.
byte[]	norms(String arg0) Returns the byte-encoded normalization factor for the named field of every document.
int	numDocs() Returns the number of documents in this index.
static IndexReader	open(Directory directory) Overrides IndexReader.open(Directory) to create an IdleTimeoutRefreshingIndexReader.
static IdleTimeoutRefreshingIndexReader	open(Directory directory, long timeout, long refreshInterval, long waitToClose) Constructs and returns an IdleTimeoutRefreshingIndexReader.
static IndexReader	open(File file) Overrides IndexReader.open(File) to create an IdleTimeoutRefreshingIndexReader.
static IdleTimeoutRefreshingIndexReader	open(File file, long timeout, long refreshInterval, long waitToClose) Constructs and returns an IdleTimeoutRefreshingIndexReader.
static IndexReader	open(String filepath) Overrides IndexReader.open(String) to create an IdleTimeoutRefreshingIndexReader.
static IdleTimeoutRefreshingIndexReader	open(String filepath, long timeout, long refreshInterval, long waitToClose) Constructs and returns an IdleTimeoutRefreshingIndexReader.
static void	setDefaultRefreshInterval(long refreshInterval) Sets the default value for the refreshInterval property.
static void	setDefaultTimeout(long timeout) Sets the default value for the timeout property.
static void	setDefaultWaitToClose(long waitToClose) Sets the default value for the waitToClose property.
static void	setMinimumWaitToClose(long minWaitToClose) Sets the minimum value for the waitToClose property.
void	setRefreshInterval(long refreshInterval) Sets the refresh interval used to check for changes to the index being read.
void	setTimeout(long timeout) Sets the idle timeout used to close inactive readers.
void	setWaitToClose(long waitToClose) Sets the wait time used to give consumers of this reader time to finish up before the underlying files are closed.
TermDocs	termDocs() Returns an unpositioned TermDocs enumerator.
TermDocs	termDocs(Term arg0) Returns an enumeration of all the documents which contain term.
TermPositions	termPositions() Returns an unpositioned TermPositions enumerator.
TermPositions	termPositions(Term arg0) Returns an enumeration of all the documents which contain term.
TermEnum	terms() Returns an enumeration of all the terms in the index.
TermEnum	terms(Term arg0) Returns an enumeration of all terms after a given term.

Methods inherited from class org.apache.lucene.index.IndexReader

close, delete, delete, finalize, indexExists, indexExists, indexExists, isLocked, isLocked, lastModified, lastModified, lastModified, unlock

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROP_MINIMUM_WAIT_TO_CLOSE

public static final String PROP_MINIMUM_WAIT_TO_CLOSE

The name of the minimumWaitToClose property.

See Also:

Constant Field Values

PROP_WAIT_TO_CLOSE

public static final String PROP_WAIT_TO_CLOSE

The name of the waitToClose property.

See Also:

Constant Field Values

PROP_TIMEOUT

public static final String PROP_TIMEOUT

The name of the timeout property.

See Also:

Constant Field Values

PROP_REFRESH_INTERVAL

public static final String PROP_REFRESH_INTERVAL

The name of the refreshInterval property.

See Also:

Constant Field Values

DEFAULT_MINIMUM_WAIT_TO_CLOSE

public static final long DEFAULT_MINIMUM_WAIT_TO_CLOSE

The default minimum value of the waitToClose property used if no default was specified and the system property is not set.

See Also:

Constant Field Values

DEFAULT_WAIT_TO_CLOSE

public static final long DEFAULT_WAIT_TO_CLOSE

The default value of the waitToClose property. used if no default was specified and the system property is not set.

See Also:

Constant Field Values

DEFAULT_TIMEOUT

public static final long DEFAULT_TIMEOUT

The default value of the timeout property. used if no default was specified and the system property is not set.

See Also:

Constant Field Values

DEFAULT_REFRESH_INTERVAL

public static final long DEFAULT_REFRESH_INTERVAL

The default value of the refreshInterval property. used if no default was specified and the system property is not set.

See Also:

Constant Field Values

Constructor Detail

IdleTimeoutRefreshingIndexReader

protected IdleTimeoutRefreshingIndexReader(Directory directory,
                                           long timeout,
                                           long refreshInterval,
                                           long waitToClose)

Constructs an IdleTimeoutRefreshingIndexReader.

Parameters:

directory - The directory to pass to the IndexReader implementation.

timeout - The idle timeout in milliseconds.

refreshInterval - The refresh interval in milliseconds.

waitToClose - The time in milliseconds to wait before closing a timed out or refreshed index.

Method Detail

open

public static IdleTimeoutRefreshingIndexReader open(Directory directory,
                                                    long timeout,
                                                    long refreshInterval,
                                                    long waitToClose)

Constructs and returns an IdleTimeoutRefreshingIndexReader.

Parameters:

directory - The directory to pass to the IndexReader implementation.

timeout - The idle timeout in milliseconds.

refreshInterval - The refresh interval in milliseconds.

waitToClose - The time in milliseconds to wait before closing a timed out or refreshed index.

Returns:

The configured idle timeout, refreshing reader.

open

public static IndexReader open(Directory directory)

Overrides IndexReader.open(Directory) to create an IdleTimeoutRefreshingIndexReader. Uses the default timings.

Parameters:

directory - The directory to pass to the IndexReader implementation.

Returns:

The configured idle timeout, refreshing reader.

open

public static IdleTimeoutRefreshingIndexReader open(File file,
                                                    long timeout,
                                                    long refreshInterval,
                                                    long waitToClose)
                                             throws IOException

Constructs and returns an IdleTimeoutRefreshingIndexReader.

Parameters:

file - The File that points to the index to read.

timeout - The idle timeout in milliseconds.

refreshInterval - The refresh interval in milliseconds.

waitToClose - The time in milliseconds to wait before closing a timed out or refreshed index.

Returns:

The configured idle timeout, refreshing reader.

IOException

open

public static IndexReader open(File file)
                        throws IOException

Overrides IndexReader.open(File) to create an IdleTimeoutRefreshingIndexReader. Uses the default timings.

Parameters:

file - The File that points to the index to read.

Returns:

The configured idle timeout, refreshing reader.

IOException

open

public static IdleTimeoutRefreshingIndexReader open(String filepath,
                                                    long timeout,
                                                    long refreshInterval,
                                                    long waitToClose)
                                             throws IOException

Constructs and returns an IdleTimeoutRefreshingIndexReader.

Parameters:

filepath - The path that points to the index to read.

timeout - The idle timeout in milliseconds.

refreshInterval - The refresh interval in milliseconds.

waitToClose - The time in milliseconds to wait before closing a timed out or refreshed index.

Returns:

The configured idle timeout, refreshing reader.

IOException

open

public static IndexReader open(String filepath)
                        throws IOException

Overrides IndexReader.open(String) to create an IdleTimeoutRefreshingIndexReader. Uses the default timings.

Parameters:

filepath - The path that points to the index to read.

Returns:

The configured idle timeout, refreshing reader.

IOException

isIdle

public boolean isIdle()

Checks if the reader has been idle longer than the idle timeout.

Returns:

true if the reader is idle.

isStale

public boolean isStale()

Checks if the index has changed since it was opened.

Returns:

true if the index has changed. false if the index has not changed or an IOException prevented verification of the index's last modified time.

doClose

protected void doClose()

Closes up any monitoring threads and the reader delegate if they exist. Doesn't throw IOException because it is caught and handled in the ReaderCloser thread if it occurrs.

Overrides:

doClose in class IndexReader

See Also:

IndexReader#doClose()

docFreq

public int docFreq(Term arg0)
            throws IOException

Description copied from class: IndexReader

Returns the number of documents containing the term t.

Specified by:

docFreq in class IndexReader

IOException

See Also:

IndexReader.docFreq(Term)

document

public Document document(int arg0)
                  throws IOException

Description copied from class: IndexReader

Returns the stored fields of the n^th Document in this index.

Specified by:

document in class IndexReader

IOException

See Also:

IndexReader.document(int)

getFieldNames

public Collection getFieldNames()
                         throws IOException

Description copied from class: IndexReader

Returns a list of all unique field names that exist in the index pointed to by this IndexReader.

Specified by:

getFieldNames in class IndexReader

Returns:

Collection of Strings indicating the names of the fields

Throws:

IOException - if there is a problem with accessing the index

See Also:

IndexReader.getFieldNames()

getFieldNames

public Collection getFieldNames(boolean arg0)
                         throws IOException

Description copied from class: IndexReader

Returns a list of all unique field names that exist in the index pointed to by this IndexReader. The boolean argument specifies whether the fields returned are indexed or not.

Specified by:

getFieldNames in class IndexReader

Parameters:

arg0 - true if only indexed fields should be returned; false if only unindexed fields should be returned.

Returns:

Collection of Strings indicating the names of the fields

Throws:

IOException - if there is a problem with accessing the index

See Also:

IndexReader.getFieldNames(boolean)

isDeleted

public boolean isDeleted(int arg0)

Description copied from class: IndexReader

Returns true if document n has been deleted

Specified by:

isDeleted in class IndexReader

Throws:

RuntimeException - if the method is accessed while the delegate reader is not open.

See Also:

IndexReader.isDeleted(int)

maxDoc

public int maxDoc()

Description copied from class: IndexReader

Returns one greater than the largest possible document number. This may be used to, e.g., determine how big to allocate an array which will have an element for every document number in an index.

Specified by:

maxDoc in class IndexReader

Throws:

RuntimeException - if the method is accessed while the delegate reader is not open.

See Also:

IndexReader.maxDoc()

norms

public byte[] norms(String arg0)
             throws IOException

Description copied from class: IndexReader

Returns the byte-encoded normalization factor for the named field of every document. This is used by the search code to score documents.

Specified by:

norms in class IndexReader

IOException

See Also:

IndexReader.norms(String)

numDocs

public int numDocs()

Description copied from class: IndexReader

Returns the number of documents in this index.

Specified by:

numDocs in class IndexReader

Throws:

RuntimeException - if the method is accessed while the delegate reader is not open.

See Also:

IndexReader.numDocs()

termDocs

public TermDocs termDocs()
                  throws IOException

Description copied from class: IndexReader

Returns an unpositioned TermDocs enumerator.

Specified by:

termDocs in class IndexReader

IOException

See Also:

IndexReader.termDocs()

termDocs

public TermDocs termDocs(Term arg0)
                  throws IOException

Description copied from class: IndexReader

Returns an enumeration of all the documents which contain term. For each document, the document number, the frequency of the term in that document is also provided, for use in search scoring. Thus, this method implements the mapping:

Term => <docNum, freq>^*

The enumeration is ordered by document number. Each document number is greater than all that precede it in the enumeration.

Overrides:

termDocs in class IndexReader

IOException

See Also:

IndexReader.termDocs(Term)

termPositions

public TermPositions termPositions()
                            throws IOException

Description copied from class: IndexReader

Returns an unpositioned TermPositions enumerator.

Specified by:

termPositions in class IndexReader

IOException

See Also:

IndexReader.termPositions()

termPositions

public TermPositions termPositions(Term arg0)
                            throws IOException

Description copied from class: IndexReader

Returns an enumeration of all the documents which contain term. For each document, in addition to the document number and frequency of the term in that document, a list of all of the ordinal positions of the term in the document is available. Thus, this method implements the mapping:

Term => <docNum, freq, <pos₁, pos₂, ... pos_freq-1> >^*

This positional information faciliates phrase and proximity searching.

The enumeration is ordered by document number. Each document number is greater than all that precede it in the enumeration.

Overrides:

termPositions in class IndexReader

IOException

See Also:

IndexReader.termPositions(Term)

terms

public TermEnum terms()
               throws IOException

Description copied from class: IndexReader

Returns an enumeration of all the terms in the index. The enumeration is ordered by Term.compareTo(). Each term is greater than all that precede it in the enumeration.

Specified by:

terms in class IndexReader

IOException

See Also:

IndexReader.terms()

terms

public TermEnum terms(Term arg0)
               throws IOException

Description copied from class: IndexReader

Returns an enumeration of all terms after a given term. The enumeration is ordered by Term.compareTo(). Each term is greater than all that precede it in the enumeration.

Specified by:

terms in class IndexReader

IOException

See Also:

IndexReader.terms(Term)

doDelete

protected void doDelete(int arg0)
                 throws IOException

Overrides:

doDelete in class IndexReader

IOException

See Also:

IndexReader#doDelete(int)

getWaitToClose

public long getWaitToClose()

Gets the wait time used to give consumers of this reader time to finish up before the underlying files are closed.

Returns:

The time to wait in milliseconds.

setWaitToClose

public void setWaitToClose(long waitToClose)

Sets the wait time used to give consumers of this reader time to finish up before the underlying files are closed.

If the time is less than getMinimumWaitToClose(), getMinimumWaitToClose() will be used instead.

Parameters:

waitToClose - The time to wait in milliseconds.

getRefreshInterval

public long getRefreshInterval()

Gets the refresh interval used to check for changes to the index being read. This is the time between checks.

Returns:

The time to wait in milliseconds.

setRefreshInterval

public void setRefreshInterval(long refreshInterval)

Sets the refresh interval used to check for changes to the index being read. This is the time between checks.

Setting this value to 0 will prevent monitoring for refreshes.

This number should be reasonably high to prevent constant checking of file modification times on the disk.

Parameters:

refreshInterval - The time to wait in milliseconds.

getTimeout

public long getTimeout()

Gets the idle timeout used to close inactive readers.

Returns:

The time to wait in milliseconds.

setTimeout

public void setTimeout(long timeout)

Sets the idle timeout used to close inactive readers.

Setting this value to 0 will prevent monitoring for idle readers.

This number should be reasonably high to prevent constant opening and closing of files on the disk.

Parameters:

timeout - The time to wait in milliseconds.

getMinimumWaitToClose

public static long getMinimumWaitToClose()

Gets the minimum value for the waitToClose property.

If this property has not been set, the system property with the key PROP_MINIMUM_WAIT_TO_CLOSE will be checked.

If the system property is not set or cannot be parsed, DEFAULT_MINIMUM_WAIT_TO_CLOSE will be used.

Returns:

The minimum time in milliseconds.

See Also:

Long#getLong(String,String)

setMinimumWaitToClose

public static void setMinimumWaitToClose(long minWaitToClose)

Sets the minimum value for the waitToClose property.

Parameters:

minWaitToClose - The minimum time in milliseconds.

getDefaultWaitToClose

public static long getDefaultWaitToClose()

Gets the default value for the waitToClose property.

If this property has not been set, the system property with the key PROP_WAIT_TO_CLOSE will be checked.

If the system property is not set or cannot be parsed, DEFAULT_WAIT_TO_CLOSE will be used.

Returns:

The default time in milliseconds.

See Also:

Long#getLong(String,String)

setDefaultWaitToClose

public static void setDefaultWaitToClose(long waitToClose)

Sets the default value for the waitToClose property.

Parameters:

waitToClose - The default time in milliseconds.

getDefaultRefreshInterval

public static long getDefaultRefreshInterval()

Gets the default value for the refreshInterval property.

If this property has not been set, the system property with the key PROP_REFRESH_INTERVAL will be checked.

If the system property is not set or cannot be parsed, DEFAULT_REFRESH_INTERVAL will be used.

Returns:

The default time in milliseconds.

See Also:

Long#getLong(String,String)

setDefaultRefreshInterval

public static void setDefaultRefreshInterval(long refreshInterval)

Sets the default value for the refreshInterval property.

Parameters:

refreshInterval - The default time in milliseconds.

getDefaultTimeout

public static long getDefaultTimeout()

Gets the default value for the timeout property.

If this property has not been set, the system property with the key PROP_TIMEOUT will be checked.

If the system property is not set or cannot be parsed, DEFAULT_TIMEOUT will be used.

Returns:

The default time in milliseconds.

See Also:

Long#getLong(String,String)

setDefaultTimeout

public static void setDefaultTimeout(long timeout)

Sets the default value for the timeout property.

Parameters:

timeout - The default time in milliseconds.

getLastAccessed

public long getLastAccessed()

Gets the last accessed time for this reader.

Returns:

The time this index was last accessed by this reader.

See Also:

getModifiedTimeOnOpen

public long getModifiedTimeOnOpen()

Gets the time this index reported as its last modified time when it was opened.

Returns:

The time this index was modified at the time it was opened.

See Also: