Critical Description
The .lock files in the state store directories do not seem to be having their file handles freed (despite the locks being freed), so on a complex job the number of file handles in use goes up rapidly as the locks are taken for the cleanup routine at the end of the thread run loop. Running lsof shows the number of open filehandles on the .lock file increasing rapidly over time. In a separate test project, I reproduced the issue and determined that in order for the filehandle to be relinquished the FileChannel instance must be properly closed.
PR: