[SPARK-15714] Fix Flaky Test: o.a.s.scheduler.BlacklistIntegrationSuite

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: Scheduler, Spark Core
• Assignee: Imran Rashid
• Reporter: Imran Rashid
• Created: 01/Jun/16 21:20
• Updated: 17/May/20 17:46
• Resolved: 03/Jun/16 16:50

Description

BlacklistIntegrationSuite (introduced by SPARK-10372) is a bit flaky because of some race conditions:

1. Failed jobs might have non-empty results, because the resultHandler will be invoked for successful tasks (if there are task successes before failures)

2. taskScheduler.taskIdToTaskSetManager must be protected by a lock on taskScheduler

Issue Links

• links to: [Github] Pull Request #13454 (squito)