[SPARK-23237] Add UI / endpoint for threaddumps for executors with active tasks

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.0
• Component/s: Web UI
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Imran Rashid
• Created: 26/Jan/18 16:27
• Updated: 08/Oct/19 05:41
• Resolved: 08/Oct/19 05:41

Description

Frequently, when there are a handful of straggler tasks, users want to know what is going on in those executors running the stragglers. Currently, that is a bit of a pain to do: you have to go to the page for your active stage, find the task, figure out which executor its on, then go to the executors page, and get the thread dump. Or maybe you just go to the executors page, find the executor with an active task, and then click on that, but that doesn't work if you've got multiple stages running.

Users could figure this by extracting the info from the stage rest endpoint, but it's such a common thing to do that we should make it easy.

I realize that figuring out a good way to do this is a little tricky. We don't want to make it easy to end up pulling thread dumps from 1000 executors back to the driver. So we've got to come up with a reasonable heuristic for choosing which executors to poll. And we've also got to find a suitable place to put this.

My suggestion is that the stage page always has a link to the thread dumps for the one executor with the longest running task. And there would be a corresponding endpoint in the rest api with the same info, maybe at /applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/slowestTaskThreadDump.

Issue Links

• is related to: SPARK-23235 Add executor Threaddump to api