[OAK-9173] Oak-run indexing fails with "This map is closed"

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.34.0
• Component/s: indexing, oak-run
• Assignee: Thomas Mueller
• Reporter: Thomas Mueller
• Created: 12/Aug/20 12:53
• Updated: 10/Sep/20 15:45
• Resolved: 28/Aug/20 06:56

Description

Near the very end of the indexing, the log file contains the following error message

06:21:13.704 [main] INFO  o.a.j.oak.plugins.index.IndexUpdate - Reindexing Traversed #13370000 /content/... [9179.83 nodes/s, 33047386.33 nodes/hr] (Elapsed 19.05 min, Expected 0.000 ns, Completed 99.95%)
06:21:14.524 [main] INFO  o.a.j.o.i.i.d.f.l.PersistedLinkedList - Entries: 1 map size: 1 file size: 9838592 bytes
06:21:17.463 [main] INFO  o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits 467781832 misses 836251
06:21:17.463 [main] INFO  o.a.j.o.i.i.d.f.l.PersistedLinkedList - Cache hits 467781980 misses 836251
06:21:24.400 [main] ERROR com.adobe.granite.indexing.tool.Main - Can't perform operation
java.lang.IllegalStateException: This map is closed [1.4.194/4]
	at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765)
	at org.h2.mvstore.MVMap.beforeWrite(MVMap.java:1044)
	at org.h2.mvstore.MVMap.remove(MVMap.java:542)
	at org.apache.jackrabbit.oak.index.indexer.document.flatfile.linkedList.PersistedLinkedList.remove(PersistedLinkedList.java:140)
	at org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNextEntry(FlatFileStoreIterator.java:102)
	at org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:85)
	at org.apache.jackrabbit.oak.index.indexer.document.flatfile.FlatFileStoreIterator.computeNext(FlatFileStoreIterator.java:39)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.index.indexer.document.DocumentStoreIndexer.reindex(DocumentStoreIndexer.java:123)

The root cause seems to be:

• The "main" iterator reaches the end, so that the map, and the file, is closed.
• But then, a "clone of the iterator" (not sure if that's the right word) is still open and tries to continue.
• This fails because the map is marked closed.

We need to keep the map + file open for a longer time.