[SPARK-6650] ExecutorAllocationManager never stops

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.1, 1.4.0
• Component/s: Spark Core
• Target Version/s: 1.3.1, 1.4.0
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 01/Apr/15 16:47
• Updated: 03/Apr/15 02:51
• Resolved: 03/Apr/15 02:51

Description

ExecutorAllocationManager doesn't even have a stop() method. That means that when the owning SparkContext goes away, the internal thread it uses to schedule its activities remains alive.

That means it constantly spams the logs and does who knows what else that could affect any future contexts that are allocated.

It's particularly evil during unit tests, since it slows down everything else after the suite is run, leaving multiple threads behind.

Issue Links

• links to: [Github] Pull Request #5311 (vanzin)