[SPARK-4955] Dynamic allocation doesn't work in YARN cluster mode

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.3.0
• Component/s: YARN
• Target Version/s: 1.3.0
• Assignee: Lianhui Wang
• Reporter: Chengxiang Li
• Created: 24/Dec/14 09:16
• Updated: 28/Jan/15 20:54
• Resolved: 28/Jan/15 20:54

Description

With executor dynamic scaling enabled, in yarn-cluster mode, after query finished and spark.dynamicAllocation.executorIdleTimeout interval, executor number is not reduced to configured min number.

Issue Links

• is depended upon by: SPARK-3145 Hive on Spark umbrella
• links to: [Github] Pull Request #3962 (lianhuiwang)