Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9761

ViewFileSystem#rename fails when using DistributedFileSystem

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.2.0
    • Component/s: viewfs
    • Labels:
      None
    • Target Version/s:

      Description

      ViewFileSystem currently passes unqualified paths (no scheme or authority) to underlying FileSystems when doing a rename. DistributedFileSystem symlink support added in HADOOP-9418 needs to qualify and check rename sources and destinations since cross-filesystem renames aren't supported, so this breaks in the following way

      • Default FS URI is configured to viewfs://<viewfs>
      • When doing a rename, ViewFileSystem checks to make sure both src and dst FileSystems are the same (which they are, both in same DFS), and then calls DistributedFileSystem#rename with unqualified "remainder" paths
      • Since these paths are unqualified, DFS qualifies them with the default FS to check that it can do the rename. This turns it into viewfs://<viewfs>/<path>
      • Since viewfs://<viewfs> is not the DFS's URI, DFS errors out the rename.

        Attachments

        1. 0001-HADOOP-9761.004.patch
          7 kB
          Colin McCabe
        2. hadoop-9761-1.patch
          5 kB
          Colin McCabe
        3. hadoop-9761-1.patch
          5 kB
          Andrew Wang
        4. hadoop-9761-2.patch
          6 kB
          Andrew Wang

          Issue Links

            Activity

              People

              • Assignee:
                andrew.wang Andrew Wang
                Reporter:
                andrew.wang Andrew Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: