Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12053

EventLog for completed applications always not found if spark.eventLog.compress is true

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.2
    • Fix Version/s: 1.5.3, 1.6.0
    • Component/s: Spark Core, Web UI
    • Labels:
      None
    • Environment:

      single AWS EC2 instance with Ubuntu
      spark standalone mode

      Description

      EventLog for completed applications always not found with following settings in spark-defaults.conf :

      spark.eventLog.enabled          true
      spark.eventLog.compress         true
      spark.eventLog.dir              file:///tmp/spark-events
      

      On Web UI, it shows always "No event logs found for application app-2015xxxxxxx"

      but there are files under /tmp/spark-events/ , such as:
      app-20151130143343-0000.snappy
      app-20151130144008-0001.snappy.inprogress
      app-20151130152905-0002.snappy.inprogress
      app-20151130153215-0003.snappy

        Attachments

          Activity

            People

            • Assignee:
              chutium Teng Qiu
              Reporter:
              chutium Teng Qiu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: