Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-961

dfs_readdir incorrectly parses paths

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2, 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: fuse-dfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      fuse-dfs dfs_readdir assumes that DistributedFileSystem#listStatus returns Paths with the same scheme/authority as the dfs.name.dir used to connect. If NameNode.DEFAULT_PORT port is used listStatus returns Paths that have authorities without the port (see HDFS-960), which breaks the following code.

      // hack city: todo fix the below to something nicer and more maintainable but
      // with good performance
      // strip off the path but be careful if the path is solely '/'
      // NOTE - this API started returning filenames as full dfs uris
      const char *const str = info[i].mName + dfs->dfs_uri_len + path_len + ((path_len == 1 && *path == '/') ? 0 : 1);
      

      Let's make the path parsing here more robust. listStatus returns normalized paths so we can find the start of the path by searching for the 3rd slash. A more long term solution is to have hdfsFileInfo maintain a path object or at least pointers to the relevant URI components.

        Attachments

        1. hdfs-961-2.patch
          3 kB
          Eli Collins
        2. hdfs-961-1.patch
          3 kB
          Eli Collins

          Issue Links

            Activity

              People

              • Assignee:
                eli Eli Collins
                Reporter:
                eli Eli Collins
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: