[SPARK-26100] [History server ]Jobs table and Aggregate metrics table are showing lesser number of tasks

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.2
• Fix Version/s: 2.4.1, 3.0.0
• Component/s: Spark Core, Web UI
• Assignee: shahid
• Reporter: ABHISHEK KUMAR GUPTA
• Created: 17/Nov/18 11:26
• Updated: 29/Nov/18 19:35
• Resolved: 26/Nov/18 21:19

Description

Test step to reproduce:

1) bin/spark-shell --master yarn --conf spark.executor.instances=3

2){{sc.parallelize(1 to 10000, 10).map

{ x => throw new RuntimeException("Bad executor")}

.collect() }}

 

3) Open Application from the history server UI

Jobs table and Aggregated metrics are showing lesser number of tasks.

 

 

Issue Links

• links to: [Github] Pull Request #23038 (shahidki31)