[HDFS-14117] RBF: We can only delete the files or dirs of one subcluster in a cluster with multiple subclusters when trash is enabled

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Labels: RBF
• Assignee: Ramkumar Ramalingam
• Reporter: Ramkumar Ramalingam
• Created: 30/Nov/18 05:26
• Updated: 06/Sep/22 08:30

Description

When we delete files or dirs in hdfs, it will move the deleted files or dirs to trash by default.

But in the global path we can only mount one trash dir /user. So we mount trash dir /user of the subcluster ns1 to the global path /user. Then we can delete files or dirs of ns1, but when we delete the files or dirs of another subcluser, such as hacluster, it will be failed.

Mount Table

Global path	Target nameservice	Target path	Order	Read only	Owner	Group	Permission	Quota/Usage	Date Modified	Date Created
/test	hacluster2	/test			securedn	users	rwxr-xr-x	[NsQuota: -/-, SsQuota: -/-]	2018/11/29 14:37:42	2018/11/29 14:37:42
/tmp	hacluster1	/tmp			securedn	users	rwxr-xr-x	[NsQuota: -/-, SsQuota: -/-]	2018/11/29 14:37:05	2018/11/29 14:37:05
/user	hacluster2,hacluster1	/user	HASH		securedn	users	rwxr-xr-x	[NsQuota: -/-, SsQuota: -/-]	2018/11/29 14:42:37	2018/11/29 14:38:20

commands: 

1./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -ls /test/.
18/11/30 11:00:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
-rw-r--r-- 3 securedn supergroup 8081 2018-11-30 10:56 /test/hdfs.cmd

2./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -ls /tmp/.
18/11/30 11:00:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
-rw-r--r--   3 securedn supergroup       6311 2018-11-30 10:57 /tmp/mapred.cmd

3../opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -rm /tmp/mapred.cmd
18/11/30 11:01:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
rm: Failed to move to trash: hdfs://router/tmp/mapred.cmd: rename destination parent /user/securedn/.Trash/Current/tmp/mapred.cmd not found.

4./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -rm /test/hdfs.cmd
18/11/30 11:01:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/11/30 11:01:22 INFO fs.TrashPolicyDefault: Moved: 'hdfs://router/test/hdfs.cmd' to trash at: hdfs://router/user/securedn/.Trash/Current/test/hdfs.cmd

Issue Links

• contains: HDFS-14430 RBF: Fix MockNamenode bug about mocking RPC getListing and mkdir
• is related to: HDFS-14343 RBF: Fix renaming folders spread across multiple subclusters
• is related to: HDFS-15083 Add new trash rpc which move the trash (mkdir and the rename) operation to the server side.
• relates to: HDFS-14431 RBF: Rename with multiple subclusters should fail if no eligible locations