[SPARK-12053] EventLog for completed applications always not found if spark.eventLog.compress is true

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.2
• Fix Version/s: 1.5.3, 1.6.0
• Component/s: Spark Core, Web UI

Environment

single AWS EC2 instance with Ubuntu
spark standalone mode

• Assignee: Teng Qiu
• Reporter: Teng Qiu
• Created: 30/Nov/15 18:41
• Updated: 30/Nov/15 22:52
• Resolved: 30/Nov/15 22:38

Description

EventLog for completed applications always not found with following settings in spark-defaults.conf :

spark.eventLog.enabled          true
spark.eventLog.compress         true
spark.eventLog.dir              file:///tmp/spark-events

On Web UI, it shows always "No event logs found for application app-2015xxxxxxx"

but there are files under /tmp/spark-events/ , such as:
app-20151130143343-0000.snappy
app-20151130144008-0001.snappy.inprogress
app-20151130152905-0002.snappy.inprogress
app-20151130153215-0003.snappy

Issue Links

• links to: [Github] Pull Request #10044 (chutium)