Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10019 make symlinks production-ready
  3. HADOOP-9972

new APIs for listStatus and globStatus to deal with symlinks

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.1-beta
    • None
    • fs
    • None

    Description

      Based on the discussion in HADOOP-9912, we need new APIs for FileSystem to deal with symlinks. The issue is that code has been written which is incompatible with the existence of things which are not files or directories. For example,
      there is a lot of code out there that looks at FileStatus#isFile, and
      if it returns false, assumes that what it is looking at is a
      directory. In the case of a symlink, this assumption is incorrect.

      It seems reasonable to make the default behavior of FileSystem#listStatus and FileSystem#globStatus be fully resolving symlinks, and ignoring dangling ones. This will prevent incompatibility with existing MR jobs and other HDFS users. We should also add new versions of listStatus and globStatus that allow new, symlink-aware code to deal with symlinks as symlinks.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cmccabe Colin McCabe Assign to me
            cmccabe Colin McCabe

            Dates

              Created:
              Updated:

              Slack

                Issue deployment