After looking at this again I am pretty certain the ultimate cause is memory. There is a load of swapping going on for me, and while the jobs are generating a lot of I/O, it's swapping that is probably overwhelming the disk. An SSD would certainly do better but it isn't the issue.
On a 4-core machine this is going to try to run 6 additional JVMs with max heap size of 1.8GB. Figuring 25% Java overhead an leaving a GB for everything else on the system, this needs 14.5GB to run comfortably. That seems pretty high as a default requirement.
But I appear to be running all these successfully with a max heap size of 768M. That works comfortably with 8GB RAM, no meaningful swapping. How about making that revision to bring the default requirement down?
In fact, if you change this to use 1 JVM per core (vs 1.5) it would also squeeze onto a 4GB / 4-core machine.