Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6165

"hdfs dfs -rm -r" and "hdfs dfs -rmdir" commands can't remove empty directory

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:

      Description

      Given a directory owned by user A with WRITE permission containing an empty directory owned by user B, it is not possible to delete user B's empty directory with either "hdfs dfs -rm -r" or "hdfs dfs -rmdir". Because the current implementation requires FULL permission of the empty directory, and throws exception.

      On the other hand, on linux, "rm -r" and "rmdir" command can remove empty directory as long as the parent directory has WRITE permission (and prefix component of the path have EXECUTE permission), For the tested OSes, some prompt user asking for confirmation, some don't.

      Here's a reproduction:

      [root@vm01 ~]# hdfs dfs -ls /user/
      Found 4 items
      drwxr-xr-x   - userabc users               0 2013-05-03 01:55 /user/userabc
      drwxr-xr-x   - hdfs    supergroup          0 2013-05-03 00:28 /user/hdfs
      drwxrwxrwx   - mapred  hadoop              0 2013-05-03 00:13 /user/history
      drwxr-xr-x   - hdfs    supergroup          0 2013-04-14 16:46 /user/hive
      [root@vm01 ~]# hdfs dfs -ls /user/userabc
      Found 8 items
      drwx------   - userabc users          0 2013-05-02 17:00 /user/userabc/.Trash
      drwxr-xr-x   - userabc users          0 2013-05-03 01:34 /user/userabc/.cm
      drwx------   - userabc users          0 2013-05-03 01:06 /user/userabc/.staging
      drwxr-xr-x   - userabc users          0 2013-04-14 18:31 /user/userabc/apps
      drwxr-xr-x   - userabc users          0 2013-04-30 18:05 /user/userabc/ds
      drwxr-xr-x   - hdfs    users          0 2013-05-03 01:54 /user/userabc/foo
      drwxr-xr-x   - userabc users          0 2013-04-30 16:18 /user/userabc/maven_source
      drwxr-xr-x   - hdfs    users          0 2013-05-03 01:40 /user/userabc/test-restore
      [root@vm01 ~]# hdfs dfs -ls /user/userabc/foo/
      [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -r -skipTrash /user/userabc/foo
      rm: Permission denied: user=userabc, access=ALL, inode="/user/userabc/foo":hdfs:users:drwxr-xr-x
      

      The super user can delete the directory.

      [root@vm01 ~]# sudo -u hdfs hdfs dfs -rm -r -skipTrash /user/userabc/foo
      Deleted /user/userabc/foo
      

      The same is not true for files, however. They have the correct behavior.

      [root@vm01 ~]# sudo -u hdfs hdfs dfs -touchz /user/userabc/foo-file
      [root@vm01 ~]# hdfs dfs -ls /user/userabc/
      Found 8 items
      drwx------   - userabc users          0 2013-05-02 17:00 /user/userabc/.Trash
      drwxr-xr-x   - userabc users          0 2013-05-03 01:34 /user/userabc/.cm
      drwx------   - userabc users          0 2013-05-03 01:06 /user/userabc/.staging
      drwxr-xr-x   - userabc users          0 2013-04-14 18:31 /user/userabc/apps
      drwxr-xr-x   - userabc users          0 2013-04-30 18:05 /user/userabc/ds
      -rw-r--r--   1 hdfs    users          0 2013-05-03 02:11 /user/userabc/foo-file
      drwxr-xr-x   - userabc users          0 2013-04-30 16:18 /user/userabc/maven_source
      drwxr-xr-x   - hdfs    users          0 2013-05-03 01:40 /user/userabc/test-restore
      [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -skipTrash /user/userabc/foo-file
      Deleted /user/userabc/foo-file
      

      Using "hdfs dfs -rmdir" command:

      bash-4.1$ hadoop fs -lsr /
      lsr: DEPRECATED: Please use 'ls -R' instead.
      drwxr-xr-x   - hdfs supergroup          0 2014-03-25 16:29 /user
      drwxr-xr-x   - hdfs   supergroup          0 2014-03-25 16:28 /user/hdfs
      drwxr-xr-x   - usrabc users               0 2014-03-28 23:39 /user/usrabc
      drwxr-xr-x   - abc    abc                 0 2014-03-28 23:39 /user/usrabc/foo-empty1
      [root@vm01 usrabc]# su usrabc
      [usrabc@vm01 ~]$ hdfs dfs -rmdir /user/usrabc/foo-empty1
      rmdir: Permission denied: user=usrabc, access=ALL, inode="/user/usrabc/foo-empty1":abc:abc:drwxr-xr-x
      

        Attachments

        1. HDFS-6165.001.patch
          10 kB
          Yongjun Zhang
        2. HDFS-6165.002.patch
          31 kB
          Yongjun Zhang
        3. HDFS-6165.003.patch
          36 kB
          Yongjun Zhang
        4. HDFS-6165.004.patch
          21 kB
          Yongjun Zhang
        5. HDFS-6165.004.patch
          21 kB
          Yongjun Zhang
        6. HDFS-6165.005.patch
          24 kB
          Yongjun Zhang
        7. HDFS-6165.006.patch
          25 kB
          Yongjun Zhang
        8. HDFS-6165.006.patch
          25 kB
          Yongjun Zhang

          Activity

            People

            • Assignee:
              yzhangal Yongjun Zhang
              Reporter:
              yzhangal Yongjun Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: