Details
Major Description
With HDFS-8983, HDFS-14802 and HDFS-15243 we are able to list protected directories which cannot be deleted or renamed, provided the following is set:
fs.protected.directories: <list of directories>
dfs.protected.subdirectories.enable: true
Testing this feature out, I can see it mostly works fine, but protected non-empty folders can still be moved to the trash. In this example /dir/protected is set in fs.protected.directories, and dfs.protected.subdirectories.enable is true.
hadoop fs -ls -R /dir drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected -rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/file1 drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected/subdir1 -rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/subdir1/file1 drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected/subdir2 -rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/subdir2/file1 [hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f -skipTrash /dir/protected/subdir1 rm: Cannot delete/rename subdirectory under protected subdirectory /dir/protected [hdfs@7d67ed1af9b0 /]$ hadoop fs -mv /dir/protected/subdir1 /dir/protected/subdir1-moved mv: Cannot delete/rename subdirectory under protected subdirectory /dir/protected ** ALL GOOD SO FAR ** [hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f /dir/protected/subdir1 2020-08-26 16:54:32,404 INFO fs.TrashPolicyDefault: Moved: 'hdfs://nn1/dir/protected/subdir1' to trash at: hdfs://nn1/user/hdfs/.Trash/Current/dir/protected/subdir1 ** It moved the protected sub-dir to the trash, where it will be deleted ** ** Checking the top level dir, it is the same ** [hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f -skipTrash /dir/protected rm: Cannot delete/rename non-empty protected directory /dir/protected [hdfs@7d67ed1af9b0 /]$ hadoop fs -mv /dir/protected /dir/protected-new mv: Cannot delete/rename non-empty protected directory /dir/protected [hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f /dir/protected 2020-08-26 16:55:32,402 INFO fs.TrashPolicyDefault: Moved: 'hdfs://nn1/dir/protected' to trash at: hdfs://nn1/user/hdfs/.Trash/Current/dir/protected1598460932388
The reason for this, seems to be that "move to trash" uses a different rename method in FSNameSystem and FSDirRenameOp which avoids the DFSUtil.checkProtectedDescendants(...) in the earlier Jiras.
I believe that "move to trash" should be protected in the same way as a -skipTrash delete.