[HDFS-15540] Directories protected from delete can still be moved to the trash

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.3.1, 3.4.0
• Component/s: namenode
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 26/Aug/20 17:28
• Updated: 27/Jan/24 03:12
• Resolved: 27/Aug/20 06:15

Description

With HDFS-8983, HDFS-14802 and HDFS-15243 we are able to list protected directories which cannot be deleted or renamed, provided the following is set:

fs.protected.directories: <list of directories>
dfs.protected.subdirectories.enable: true

Testing this feature out, I can see it mostly works fine, but protected non-empty folders can still be moved to the trash. In this example /dir/protected is set in fs.protected.directories, and dfs.protected.subdirectories.enable is true.

hadoop fs -ls -R /dir

drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected
-rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/file1
drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected/subdir1
-rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/subdir1/file1
drwxr-xr-x - hdfs supergroup 0 2020-08-26 16:52 /dir/protected/subdir2
-rw-r--r-- 3 hdfs supergroup 174 2020-08-26 16:52 /dir/protected/subdir2/file1

[hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f -skipTrash /dir/protected/subdir1
rm: Cannot delete/rename subdirectory under protected subdirectory /dir/protected

[hdfs@7d67ed1af9b0 /]$ hadoop fs -mv /dir/protected/subdir1 /dir/protected/subdir1-moved
mv: Cannot delete/rename subdirectory under protected subdirectory /dir/protected

** ALL GOOD SO FAR **

[hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f /dir/protected/subdir1
2020-08-26 16:54:32,404 INFO fs.TrashPolicyDefault: Moved: 'hdfs://nn1/dir/protected/subdir1' to trash at: hdfs://nn1/user/hdfs/.Trash/Current/dir/protected/subdir1

** It moved the protected sub-dir to the trash, where it will be deleted **

** Checking the top level dir, it is the same **

[hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f -skipTrash /dir/protected 
rm: Cannot delete/rename non-empty protected directory /dir/protected

[hdfs@7d67ed1af9b0 /]$ hadoop fs -mv /dir/protected /dir/protected-new
mv: Cannot delete/rename non-empty protected directory /dir/protected

[hdfs@7d67ed1af9b0 /]$ hadoop fs -rm -r -f /dir/protected 
2020-08-26 16:55:32,402 INFO fs.TrashPolicyDefault: Moved: 'hdfs://nn1/dir/protected' to trash at: hdfs://nn1/user/hdfs/.Trash/Current/dir/protected1598460932388

The reason for this, seems to be that "move to trash" uses a different rename method in FSNameSystem and FSDirRenameOp which avoids the DFSUtil.checkProtectedDescendants(...) in the earlier Jiras.

I believe that "move to trash" should be protected in the same way as a -skipTrash delete.