[HADOOP-11601] Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 3.0.0-alpha2
• Component/s: fs, test
• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 16/Feb/15 11:19
• Updated: 17/Nov/16 22:40
• Resolved: 17/Nov/16 22:35

Description

HADOOP-11584 has shown that the contract tests are not validating that FileStatus.getBlocksize() must be >0 for any analytics jobs to partition workload correctly.

Clarify in text and add test to do this. Test MUST be designed to work against eventually consistent filesystems where getFileStatus() may not be immediately visible, by retrying operation if FS declares it is an object store.

Issue Links

• relates to: HADOOP-11584 s3a file block size set to 0 in getFileStatus
• links to: GitHub Pull Request #50