Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16140

hadoop fs expunge to add -immediate option to purge trash immediately

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.2.1, 3.1.4
    • Component/s: fs
    • Labels:
      None

      Description

      I have always felt the HDFS trash is missing a simple way to empty the current users trash immediately. We have "expunge" but in my experience supporting clusters, end users find this confusing. When most end users run expunge, they really want to empty their trash immediately and get confused when expunge does not do this.

      This can result in users performing somewhat dangerous "skipTrash" operations on the trash to free up space. The alternative, which most users will not figure out on their own is:

      1. Run the expunge command once - this will move the current folder to a checkpoint and remove any old checkpoints older than the retention interval
      2. Wait over 1 minute and then run expunge again, overriding fs.trash.interval to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 -expunge.

      With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" that purges everything in the logged in users Trash directories immediately.

      How would the community feel about adding this new option? I will upload a patch for comments.

        Attachments

        1. HDFS-14200.001.patch
          9 kB
          Stephen O'Donnell
        2. HADOOP-14200.006.patch
          12 kB
          Stephen O'Donnell
        3. HADOOP-14200.005.patch
          10 kB
          Stephen O'Donnell
        4. HADOOP-14200.004.patch
          10 kB
          Stephen O'Donnell
        5. HADOOP-14200.003.patch
          9 kB
          Stephen O'Donnell
        6. HADOOP-14200.002.patch
          9 kB
          Stephen O'Donnell

          Issue Links

            Activity

              People

              • Assignee:
                sodonnell Stephen O'Donnell
                Reporter:
                sodonnell Stephen O'Donnell
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: