The solution is to define another queue for the tasks considered as low priority, so that they don't fill the execution queue.
Then, depending on the executor's load poll this queue for additional work items.
The secondary queue will only be used as needed, and the load is configurable via the system property
This property is meant to be used as a percent. 0 means disabled / the default is 75.
There are some timing issues with the indexing tests on account of this new async text extraction. I've tried to fix all of them, but there may be more.
I haven't touched yet on the tika extraction that happens in a different process. I think that will need some minor refactoring as well.
Attaching proposed patch.