[HADOOP-9972] new APIs for listStatus and globStatus to deal with symlinks

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.1.1-beta
• Component/s: fs
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 17/Sep/13 16:16
• Updated: 04/Oct/13 02:10

Description

Based on the discussion in HADOOP-9912, we need new APIs for FileSystem to deal with symlinks. The issue is that code has been written which is incompatible with the existence of things which are not files or directories. For example,
there is a lot of code out there that looks at FileStatus#isFile, and
if it returns false, assumes that what it is looking at is a
directory. In the case of a symlink, this assumption is incorrect.

It seems reasonable to make the default behavior of FileSystem#listStatus and FileSystem#globStatus be fully resolving symlinks, and ignoring dangling ones. This will prevent incompatibility with existing MR jobs and other HDFS users. We should also add new versions of listStatus and globStatus that allow new, symlink-aware code to deal with symlinks as symlinks.

Issue Links

• blocks: HADOOP-9877 Fix listing of snapshot directories in globStatus
• is related to: HADOOP-9912 globStatus of a symlink to a directory does not report symlink as a directory
• relates to: HADOOP-9984 FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default