[OAK-5553] Index async index in a new lane without blocking the main lane

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: indexing
• Epic Link: indexer resilience
• Assignee: Unassigned
• Reporter: Chetan Mehrotra
• Created: 31/Jan/17 08:14
• Updated: 26/May/21 14:53

Description

Currently if an async index has to be reindex for any reason say update of index definition then this process blocks the indexing of other indexes on that lane.

For e.g. if on "async" lane we have 2 indexes /oak:index/fooIndex and /oak:index/barIndex and fooIndex needs to be reindexed. In such a case currently AsyncIndexUpdate would work on reindexing and untill that gets complete other index do not receive any update. If the reindexing takes say 1 day then other index would start lagging behind by that time. Note that NRT indexing would help somewhat here.

To improve this we can implement something similar to what was done for property index in OAK-1456 i.e. provide a way where

1. an admin can trigger reindex of some async indexes
2. those indexes are moved to different lane and then reindexed
3. post reindexing logic should then move them back to there original lane

Further this task can then be performed on non leader node as the indexes would not be part of any active lane. Also we may implement it as part of oak-run

Issue Links

• relates to: OAK-1456 Non-blocking reindexing