[OAK-7074] Ensure that all Documents are read with document order traversal indexing

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: mongomk, run
• Epic Link: indexer resilience
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 18/Dec/17 10:08
• Updated: 17/Mar/20 12:14

Description

With OAK-6353 support was added for document order traversal indexing. In this mode we open a DB cursor and try to read all documents from it using document order traversal. Such a cursor may remain open for long time (2-4 hrs) and its possible that document may get reordered by the Mongo storage engine. This would result in 2 aspects to be thought about

1. Duplicate documents - Same document may appear more than once in result set
2. Possibly missed document - It may be a possibility that a document got moved and missed becoming part of cursor.

Both these aspects would need to be handled

Issue Links

• relates to: OAK-6353 Use Document order traversal for reindexing performed on DocumentNodeStore setups