[SPARK-963] Races in JobLoggerSuite

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.8.1, 0.9.0
• Fix Version/s: 0.8.1
• Component/s: Spark Core
• Assignee: Patrick McFadin
• Reporter: Mark Hamstra
• Created: 24/Nov/13 15:37
• Updated: 08/Dec/13 18:24
• Resolved: 08/Dec/13 18:24

Description

Since SparkListener events are now processed asynchronously from the SparkListenerBus eventQueue, the jobLogger conditions checked in the "inner variables" and "interface functions" tests of the JobLoggerSuite are not guaranteed to be satisfied just because rdd.reduceByKey(+).collect() has returned. This leads to infrequent failures of these tests when we lose the race.