[SPARK-26760] [Spark Incorrect display in SPARK UI Executor Tab when number of cores is 4 and Active Task display as 5 in Executor Tab of SPARK UI]

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.0
• Component/s: Spark Core
• Labels: bulk-closed

Environment

Spark 2.4

• Assignee: Unassigned
• Reporter: ABHISHEK KUMAR GUPTA
• Created: 29/Jan/19 06:02
• Updated: 25/May/21 01:55
• Resolved: 25/May/21 01:44

Description

Steps:

1. Launch Spark Shell 
2. bin/spark-shell --master yarn  --conf spark.dynamicAllocation.enabled=true --conf spark.dynamicAllocation.initialExecutors=3 --conf spark.dynamicAllocation.minExecutors=1 --conf spark.dynamicAllocation.executorIdleTimeout=60s --conf spark.dynamicAllocation.maxExecutors=5
3. Submit a Job sc.parallelize(1 to 10000,116000).count()
4. Check the YARN UI Executor Tab for the RUNNING application
5. UI display as Number of cores 4 and Active Tasks column shows as 5

Expected:

It Number of Active Tasks should be same as Number of Cores.