[OAK-5434] Skip processing of queued changes if async index update is detected in ExternalIndexObserver

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: lucene
• Labels: performance
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 10/Jan/17 12:00
• Updated: 07/Nov/17 13:20

Description

ExternalIndexObserver is currently backed by a queue (its wrapped in BackgroundObserver). Currently it processed the changes one by one as received from the queue. If this processing takes long time then its possible that it would lag behind the async indexing cycle.

So ExternalIndexObserver may be busy indexing changes from [r1-r2] but async indexing is already done indexing changes upto r3 (r3 > r2) and IndexTracker would move to newer index version. In such case work done by ExternalIndexObserver is wasted.

This can be optimized by ensuring that ExternalIndexObserver can see the lastIndexTo of :async as per latest entry in queue. If that is newer than one its processing then it can skip processing the queue entry and thus free up space in queue

Issue Links

• is related to: OAK-4808 Index external changes as part of NRT indexing