Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12485

expunge may fail to remove trash from encryption zone

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 2.8.3, 3.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This is related to HDFS-12484, but turns out that even if I have super user permission, -expunge may not remove trash either.

      If I log into Linux as root, and then login as the superuser hdfs@EXAMPLE.COM

      [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
      17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      [root@nightly511-1 ~]# hdfs dfs -expunge
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: /user/hdfs/.Trash/170918143916
      17/09/18 15:21:59 INFO fs.TrashPolicyDefault: TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
      [root@nightly511-1 ~]# hdfs dfs -ls hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      -rw-r--r--   3 hdfs systest          0 2017-09-18 15:21 hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
      

      expunge does not remove trash under /scale, because it does not know I am 'hdfs' user.

      DistributedFileSystem#getTrashRoots
      Path ezTrashRoot = new Path(it.next().getPath(),
                  FileSystem.TRASH_PREFIX);
              if (!exists(ezTrashRoot)) {
                continue;
              }
              if (allUsers) {
                for (FileStatus candidate : listStatus(ezTrashRoot)) {
                  if (exists(candidate.getPath())) {
                    ret.add(candidate);
                  }
                }
              } else {
                Path userTrash = new Path(ezTrashRoot, System.getProperty(
                    "user.name")); --> bug
                try {
                  ret.add(getFileStatus(userTrash));
                } catch (FileNotFoundException ignored) {
                }
              }
      

      It should use UGI for user name, rather than system login user name.

        Attachments

          Activity

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: