Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-901

Make S3FileSystem do recursive renames

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.11.0
    • Component/s: fs
    • Labels:
      None

      Description

      From Mike Smith:

      I went through the S3FileSystem.java codes and fixed the renameRaw() method.
      Now, it iterates through the folders recursively and rename those. Also, in
      the case of existing destination folder, it moves the src folder under the
      dst folder.

      Here is the piece code that should be replaced in S3FileSystem.java.
      renameRaw() method should be replaced by the following methods:

      @Override
      public boolean renameRaw(Path src, Path dst) throws IOException {

      Path absoluteDst = makeAbsolute(dst);
      Path absoluteSrc = makeAbsolute(src);

      INode inode = store.getINode(absoluteDst);
      // checking to see of dst folder exist. In this case moves the
      // src folder under the existing path.
      if (inode != null && inode.isDirectory())

      { Path newDst = new Path(absoluteDst.toString ()+"/"+absoluteSrc.getName()); return renameRaw(src,newDst,src); }

      else

      { // if the dst folder does not exist, then the dst folder will be created. return renameRaw(src,dst,src); }

      }

      // recursively goes through all the subfolders and rename those.
      public boolean renameRaw(Path src, Path dst,Path orgSrc) throws
      IOException {
      Path absoluteSrc = makeAbsolute(src);
      Path newDst = new Path(src.toString().replaceFirst(orgSrc.toString(),
      dst.toString()));
      Path absoluteDst = makeAbsolute(newDst);
      LOG.info(absoluteSrc.toString());
      INode inode = store.getINode (absoluteSrc);
      if (inode == null)

      { return false; }

      if (inode.isFile())

      { store.storeINode(makeAbsolute(absoluteDst), inode); }

      else {
      store.storeINode (makeAbsolute(absoluteDst), inode);
      Path[] contents = listPathsRaw(absoluteSrc);
      if (contents == null)

      { return false; }

      for (Path p : contents) {
      if (! renameRaw(p,dst,orgSrc))

      { return false; }

      }
      }
      store.deleteINode(absoluteSrc);
      return true;
      }

        Attachments

        1. hadoop-901.patch
          10 kB
          Thomas White

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tomwhite Thomas White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: