Till now , every job which aborted by completed blacklist just show log like below which has no more information:
Aborting $taskSet because task $indexInTaskSet (partition $partition) cannot run anywhere due to node and executor blacklist. Blacklisting behavior cannot run anywhere due to node and executor blacklist.Blacklisting behavior can be configured via spark.blacklist.*."
We could add most recent failure reason for taskset blacklist which can be showed on spark ui to let user know failure reason directly.
An example after modifying:
Aborting TaskSet 0.0 because task 0 (partition 0) cannot run anywhere due to node and executor blacklist.
Most recent failure:
Some(Lost task 0.1 in stage 0.0 (TID 3,xxx, executor 1): java.lang.Exception: Fake error!
Blacklisting behavior can be configured via spark.blacklist.*.