Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      See spec in parent issue (SPARK-18085) for more details.

      This task is actually not explicit in the spec, and it's also an issue with the current SHS. But having the SHS persist listing data makes it worse.

      Basically, the SHS currently does not detect when files are deleted from the event log directory manually; so those applications are still listed, and trying to see their UI will either show the UI (if it's loaded) or an error (if it's not).

      With the new SHS, that also means that data is leaked in the disk stores used to persist listing and UI data, making the problem worse.

        Attachments

          Activity

            People

            • Assignee:
              vanzin Marcelo Masiero Vanzin
              Reporter:
              vanzin Marcelo Masiero Vanzin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: