Following up on this (a relatively old) thread , we should do pruning of property index structure asynchronously. The thread was never concluded.. here are a couple of ideas picked from the thread:
- Move pruning to an async thread
- Throttle pruning i.e. prune only once in a while
- I'm not sure how that would work though – an unpruned part would remain as is until another index happens on that path.
A couple of cases which have concurrent updates hitting merge conflicts in our product (Adobe AEM):
- Some index are very volatile (in the sense that indexed property switches its values very quickly) e.g. sling job status, AEM workflow status.
- Multiple threads take care of jobs. Although sling maintains a bucketed structure for job storage to reduce conflicts... but inside index tree the bucket structure, at times, gets pruned and needs to be created in the next job status change
While retries do take care of these conflict a lot of times and even when they don't, AEM workflows has it's own retry to work around. But, retrying, IMHO, is just a waste of time – more importantly in paths where application doesn't really have a control.
Yes, there'd be some left over paths in index structure between asynchronous prunes. But, I think the cost of such wasted traversals would be covered up with time saved in avoiding the concurrent update conflict.