Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15893

fs.TrashPolicyDefault: can't create trash directory and race condition

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: common
    • Labels:
      None

      Description

      there is race condition in method moveToTrash class TrashPolicyDefault

      try {
      if (!fs.mkdirs(baseTrashPath, PERMISSION))

      { // create current LOG.warn("Can't create(mkdir) trash directory: " + baseTrashPath); return false; }

      } catch (FileAlreadyExistsException e) {
      // find the path which is not a directory, and modify baseTrashPath
      // & trashPath, then mkdirs
      Path existsFilePath = baseTrashPath;
      while (!fs.exists(existsFilePath))

      { existsFilePath = existsFilePath.getParent(); }

      // case

        other thread deletes existsFilePath here ,the results doesn't  meet expectation

       for example

         there is /user/u_sunlisheng/.Trash/Current/user/u_sunlisheng/b

         when delete /user/u_sunlisheng/b/a. if existsFilePath is deleted, the result is /user/u_sunlisheng/.Trash/Current/user/u_sunlisheng+timstamp/b/a

        so  when existsFilePath is deleted, don't modify baseTrashPath.    

      baseTrashPath = new Path(baseTrashPath.toString().replace(
      existsFilePath.toString(), existsFilePath.toString() + Time.now())
      );

      trashPath = new Path(baseTrashPath, trashPath.getName());
      // retry, ignore current failure
      --i;
      continue;
      } catch (IOException e)

      { LOG.warn("Can't create trash directory: " + baseTrashPath, e); cause = e; break; }

       

        Attachments

        1. HADOOP-15893.002.patch
          3 kB
          sunlisheng
        2. HADOOP-15893.001.patch
          3 kB
          sunlisheng

          Issue Links

            Activity

              People

              • Assignee:
                leosun sunlisheng
                Reporter:
                leosun sunlisheng
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: