This looks good. Thanks for adding the unit tests too. We should check this into 0.21 as well, if that's not out yet.
The only concern I have is that the existing unit tests, such as testMinAndFairSharePreemption, work correctly. This seems to be because those pools either only test one type of preemption (min-share or fair-share), or they place the over-scheduled jobs in pools that have no min share set. This means that one of the values in max(tasksDueToMinShare, tasksDueToFairShare) is zero. Would you mind creating a second copy of testMinAndFairSharePreemption where job 1 is in a pool with a min share set (i.e. not in the default pool)?
A minor comment on clarity: rather than adding the line "tasksToPreempt = tasksToPreempt < 0 ? 0 : tasksToPreempt", it would be better to make sure that tasksDueToMinShare and tasksDueToFairShare are themselves never negative. You can do it by adding a max(0, ...) on the lines where they are computed (for example, tasksDueToMinShare = Math.max(0, target - sched.getRunningTasks())).