[SPARK-5106] Make Web UI automatically refresh/update displayed data

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.2.0
• Component/s: Web UI
• Assignee: Unassigned
• Reporter: Ryan Williams
• Created: 06/Jan/15 17:32
• Updated: 01/Dec/15 18:39
• Resolved: 01/Dec/15 18:39

Description

My (and presumably others') experience monitoring Spark jobs currently consists of repeatedly ⌘R'ing various pages of the web UI to view ever-fresher data about how many tasks have succeeded / failed, how much spillage is happening, etc., which is tedious.

Particularly unfortunate is the "one refresh over the line" problem where, just as things are getting interesting, the job itself fails or finishes, and after refreshing the page all data disappears.

It would be good if the web UI updated the data it was displaying automatically.

One hacky way to achieve this would be to have it automatically refresh the page, though this still risks losing everything when the job finishes.

A better long-term solution would be to have the UI poll for (or have pushed to it) updates to the data it is displaying.

Either way, some way to toggle this functionality on or off is probably warranted as well.

Issue Links

• links to: [Github] Pull Request #9860 (jbonofre)