[HADOOP-7352] FileSystem#listStatus should throw IOE upon access error

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 3.0.0-alpha2
• Component/s: fs
• Target Version/s: 3.0.0-alpha2
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: Change FileSystem#listStatus contract to never return null. Local filesystems prior to 3.0.0 returned null upon access error. It is considered erroneous. We should expect FileSystem#listStatus to throw IOException upon access error.
• Assignee: John Zhuge
• Reporter: Matthew Foley
• Created: 02/Jun/11 06:16
• Updated: 15/May/17 13:50
• Resolved: 19/Oct/16 01:20

Description

In HADOOP-6201 and HDFS-538 it was agreed that FileSystem::listStatus should throw FileNotFoundException instead of returning null, when the target directory did not exist.

However, in LocalFileSystem implementation today, FileSystem::listStatus still may return null, when the target directory exists but does not grant read permission. This causes NPE in many callers, for all the reasons cited in HADOOP-6201 and HDFS-538. See HADOOP-7327 and its linked issues for examples.

Issue Links

• breaks: YARN-5777 TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails
• breaks: HADOOP-13787 Azure testGlobStatusThrowsExceptionForUnreadableDir fails
• is blocked by: HADOOP-7363 TestRawLocalFileSystemContract is needed
• is depended upon by: HADOOP-12718 Incorrect error message by fs -put local dir without permission
• is duplicated by: HADOOP-6377 ChecksumFileSystem.getContentSummary throws NPE when directory contains inaccessible directories
• is duplicated by: HADOOP-13191 FileSystem#listStatus should not return null
• is related to: HADOOP-13795 Skip testGlobStatusThrowsExceptionForUnreadableDir in TestFSMainOperationsSwift
• is related to: HADOOP-7327 FileSystem.listStatus() throws NullPointerException instead of IOException upon access permission failure
• is related to: HADOOP-9361 Strictly define the expected behavior of filesystem APIs and write tests to verify compliance
• relates to: HADOOP-13373 Add S3A implementation of FSMainOperationsBaseTest
• relates to: HDFS-538 DistributedFileSystem::listStatus incorrectly returns null for empty result sets
• relates to: HDFS-303 Make contracts of LocalFileSystem and DistributedFileSystem consistent
• relates to: HADOOP-6201 FileSystem::ListStatus should throw FileNotFoundException