Details
Description
If the event logging is enabled, the Spark Standalone Master tries to recreate the web UI of a completed Spark application from its event logs. However if this event log is huge (e.g. for a Spark Streaming application), then the master hangs in its attempt to read and recreate the web ui. This hang causes the whole standalone cluster to be unusable.
Workaround is to disable the event logging.
Attachments
Issue Links
- duplicates
-
SPARK-12299 Remove history serving functionality from standalone Master
- Resolved
- Is contained by
-
SPARK-12299 Remove history serving functionality from standalone Master
- Resolved
- is duplicated by
-
SPARK-7600 Stopping Streaming Context (sometimes) crashes master
- Resolved
- relates to
-
SPARK-12108 Event logs are much bigger in 1.6 than in 1.5
- Resolved