Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12485

expunge may fail to remove trash from encryption zone

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0, 3.0.0-alpha1
    • 2.9.0, 2.8.3, 3.0.0
    • None
    • None
    • Reviewed

    Description

      This is related to HDFS-12484, but turns out that even if I have super user permission, -expunge may not remove trash either.

      If I log into Linux as root, and then login as the superuser hdfs@EXAMPLE.COM

      [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
      17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      [root@nightly511-1 ~]# hdfs dfs -expunge
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: /user/hdfs/.Trash/170918143916
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      [root@nightly511-1 ~]# hdfs dfs -ls hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      -rw-r--r--   3 hdfs systest          0 2017-09-18 15:21 hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      

      expunge does not remove trash under /scale, because it does not know I am 'hdfs' user.

      DistributedFileSystem#getTrashRoots
      Path ezTrashRoot = new Path(it.next().getPath(),
                  FileSystem.TRASH_PREFIX);
              if (!exists(ezTrashRoot)) {
                continue;
              }
              if (allUsers) {
                for (FileStatus candidate : listStatus(ezTrashRoot)) {
                  if (exists(candidate.getPath())) {
                    ret.add(candidate);
                  }
                }
              } else {
                Path userTrash = new Path(ezTrashRoot, System.getProperty(
                    "user.name")); --> bug
                try {
                  ret.add(getFileStatus(userTrash));
                } catch (FileNotFoundException ignored) {
                }
              }
      

      It should use UGI for user name, rather than system login user name.

      Attachments

        1. HDFS-12485.001.patch
          2 kB
          Wei-Chiu Chuang

        Activity

          People

            weichiu Wei-Chiu Chuang
            weichiu Wei-Chiu Chuang
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: