[OAK-5924] Prevent long running query from delaying refresh of index

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: lucene
• Epic Link: indexer resilience
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 14/Mar/17 05:58
• Updated: 26/May/21 14:52

Description

Whenever the index gets updated IndexTracker detects the changes and open new IndexNode and closes old index nodes. This flow would block untill all old IndexNode are closed.

IndexNode close itself relies on a writer lock. It can happen that a long running query i.e. a query which is about to read a page of large is currently executing on the old IndexNode instance. This query is trying load 100k docs and is very slow (due to loading of excerpt) then such a query would prevent the IndexNode from getting closed. This in turn would prevent the index from seeing latest data and become stale.

To make query and indexing more resilient we should look if current IndexNode being used for query is closing or not. If closing then query should open a fresh searcher