[SPARK-4893] Clean up uses of System.setProperty in unit tests

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Spark Core
• Assignee: Josh Rosen
• Reporter: Josh Rosen
• Created: 19/Dec/14 01:44
• Updated: 30/Dec/14 19:54
• Resolved: 30/Dec/14 19:54

Description

Several of our tests call System.setProperty (or test code which implicitly sets system properties) and don't always reset/clear the modified properties, which can create ordering dependencies between tests and cause hard-to-diagnose failures.

I think we can remove most of these System.setProperty calls and replace them by proper usage of SparkConf.

For the remaining cases where we still need to use System.setProperty, I think that we should use a ScalaTest fixture / mixin to automatically snapshot and restore system properties after test runs.

Issue Links

• duplicates: SPARK-1010 Update all unit tests to use SparkConf instead of system properties
• links to: [Github] Pull Request #3739 (JoshRosen)