Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5658

YARN should have a hook to delete a path from HDFS when an application ends

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      There are many cases when a client uploads data to HDFS and then needs to subsequently clean it up, such as with the distributed cache. It would be helpful if YARN would do that cleanup automatically on job completion.

      The hook could be generic to an URI supported by FileSystem.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                templedf Daniel Templeton
                Reporter:
                templedf Daniel Templeton
              • Votes:
                1 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: