Minor Description
Indexes are merged using the configuration parameters mergeFactor and minMergeDocs. With the default value of 10 for mergeFactor and 100 for minMergeDocs, as soon as 10 index directories exist with less or equal than 100 nodes they are merged into a single one. This process is then repeated by multiplying the minMergeDocs with the mergeFactor. Therefore the second round will merge 10 index directories with less or equal than 1000 nodes.
Because the above process is part of the regular workspace store operation an index merge with more than let's say 10'000 nodes can block the store operation for a couple of seconds. With the current synchronization scheme, all other threads are blocked from writing. This is not acceptable.
Index merging should run in a separate thread in the background.
The process needs to take care of the following:
- While merging indexes, deletes on those indexes must not get lost
- Switching between the indexes that are merged and the new index must be atomic
- Recovery if merging is interrupted, e.g. jackrabbit is shutdown