[SPARK-3490] Alleviate port collisions during tests

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 0.9.3, 1.1.1, 1.2.0
• Component/s: Spark Core
• Target Version/s: 0.9.3, 1.0.3, 1.1.1, 1.2.0
• Assignee: Andrew Or
• Reporter: Andrew Or
• Created: 11/Sep/14 16:42
• Updated: 16/May/15 11:56
• Resolved: 16/May/15 11:56

Description

A few tests, notably SparkSubmitSuite and DriverSuite, have been failing intermittently because we open too many ephemeral ports and occasionally can't bind to new ones.

We should minimize the use of ports during tests where possible. A great candidate is the SparkUI, which is not needed for most tests.

Issue Links

• duplicates: SPARK-1047 Ability to disable the spark ui server (unit tests)
• relates to: SPARK-3431 Parallelize Scala/Java test execution
• links to: [Github] Pull Request #2363 (andrewor14)
• links to: [Github] Pull Request #2415 (andrewor14)
• links to: [Github] Pull Request #3959 (andrewor14)
• links to: [Github] Pull Request #3961 (andrewor14)