[SPARK-12299] Remove history serving functionality from standalone Master

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 2.0.0
• Component/s: Deploy
• Target Version/s: 2.0.0
• Assignee: Bryan Cutler
• Reporter: Andrew Or
• Created: 12/Dec/15 01:31
• Updated: 09/May/18 20:32
• Resolved: 04/May/16 21:34

Description

The standalone Master currently continues to serve the historical UIs of applications that have completed and enabled event logging. This poses problems, however, if the event log is very large, e.g. SPARK-6270. The Master might OOM or hang while it rebuilds the UI, rejecting applications in the mean time.

Personally, I have had to make modifications in the code to disable this myself, because I wanted to use event logging in standalone mode for applications that produce a lot of logging.

Removing this from the Master would simplify the process significantly. This issue supersedes SPARK-12062.

Issue Links

• contains: SPARK-6270 Standalone Master hangs when streaming job completes and event logging is enabled
• is duplicated by: SPARK-6270 Standalone Master hangs when streaming job completes and event logging is enabled
• supercedes: SPARK-12062 Master rebuilding historical SparkUI should be asynchronous
• links to: [Github] Pull Request #10991 (BryanCutler)