Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9761

ViewFileSystem#rename fails when using DistributedFileSystem

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.1.0-beta, 3.0.0-alpha1
    • 2.2.0
    • viewfs
    • None

    Description

      ViewFileSystem currently passes unqualified paths (no scheme or authority) to underlying FileSystems when doing a rename. DistributedFileSystem symlink support added in HADOOP-9418 needs to qualify and check rename sources and destinations since cross-filesystem renames aren't supported, so this breaks in the following way

      • Default FS URI is configured to viewfs://<viewfs>
      • When doing a rename, ViewFileSystem checks to make sure both src and dst FileSystems are the same (which they are, both in same DFS), and then calls DistributedFileSystem#rename with unqualified "remainder" paths
      • Since these paths are unqualified, DFS qualifies them with the default FS to check that it can do the rename. This turns it into viewfs://<viewfs>/<path>
      • Since viewfs://<viewfs> is not the DFS's URI, DFS errors out the rename.

      Attachments

        1. hadoop-9761-2.patch
          6 kB
          Andrew Wang
        2. hadoop-9761-1.patch
          5 kB
          Andrew Wang
        3. hadoop-9761-1.patch
          5 kB
          Colin McCabe
        4. 0001-HADOOP-9761.004.patch
          7 kB
          Colin McCabe

        Issue Links

          Activity

            People

              andrew.wang Andrew Wang
              andrew.wang Andrew Wang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: