[HADOOP-2565] DFSPath cache of FileStatus can become stale

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.18.0
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Doug Cutting
• Created: 09/Jan/08 18:53
• Updated: 02/May/13 02:29
• Resolved: 05/Jun/08 20:58

Description

Paths returned from DFS internally cache their FileStatus, so that getStatus(Path) does not require another RPC. This cache is never refreshed and become stale, resulting in program error.

This should not be fixed until FileSystem#listStatus() is removed by HADOOP-2563, and user code is thus no longer dependent on this cache for good performance.

Issue Links

• blocks: HADOOP-1700 Append to files in HDFS
• depends upon: HADOOP-2563 Remove deprecated FileSystem#listPaths()
• depends upon: HADOOP-3095 Validating input paths and creating splits is slow on S3
• is blocked by: HADOOP-2566 need FileSystem#globStatus method