Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0
    • Component/s: agent
    • Labels:
      None

      Description

      Currently, there is no garbage collection of completed executors/frameworks on slaves, leading to local directories periodically filling up.

      Design proposal:

      • A libprocess process that gets messages when an executor has gotten cleaned up and sets some timeout when it should clean up the directory. Abstracting it this way is nice because we could add other capabilities like copying the directory output to HDFS if requested by the user.
      • Making sure directories continue to get cleaned up after a slave restarts
      • Make sure frameworks use their executor's working directory for storage rather than a hard-configured path (e.g. /tmp)

        Attachments

          Activity

            People

            • Assignee:
              vinodkone Vinod Kone
              Reporter:
              twm378 Thomas Marshall
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 720h
                720h
                Remaining:
                Remaining Estimate - 720h
                720h
                Logged:
                Time Spent - Not Specified
                Not Specified