Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-961

dfs_readdir incorrectly parses paths

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1, 0.20.2, 0.21.0
    • 0.21.0
    • fuse-dfs
    • None
    • Reviewed

    Description

      fuse-dfs dfs_readdir assumes that DistributedFileSystem#listStatus returns Paths with the same scheme/authority as the dfs.name.dir used to connect. If NameNode.DEFAULT_PORT port is used listStatus returns Paths that have authorities without the port (see HDFS-960), which breaks the following code.

      // hack city: todo fix the below to something nicer and more maintainable but
      // with good performance
      // strip off the path but be careful if the path is solely '/'
      // NOTE - this API started returning filenames as full dfs uris
      const char *const str = info[i].mName + dfs->dfs_uri_len + path_len + ((path_len == 1 && *path == '/') ? 0 : 1);
      

      Let's make the path parsing here more robust. listStatus returns normalized paths so we can find the start of the path by searching for the 3rd slash. A more long term solution is to have hdfsFileInfo maintain a path object or at least pointers to the relevant URI components.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            eli Eli Collins Assign to me
            eli Eli Collins
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment