[HADOOP-17323] s3a getFileStatus("/") to skip IO

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.1
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Mukund Thakur
• Reporter: Steve Loughran
• Created: 22/Oct/20 18:24
• Updated: 14/Dec/20 17:22
• Resolved: 24/Nov/20 11:25

Description

Calls to "hadoop fs ls s3a://something/" add a full getFileStatus() sequence (HEAD, LIST, LIST) because FsShell Ls calls getFileStatus on every input.

We should just build a root status entry immediately. There's one consequence: if a user has disabled the s3a existence probe, there will be no checks for the bucket existence. But that can be viewed as a "well, that's what happens" behaviour

Issue Links

• breaks: HADOOP-17433 Skipping network I/O in S3A getFileStatus(/) breaks ITestAssumeRole
• causes: HADOOP-17398 Skipping network I/O in S3A getFileStatus(/) breaks some tests
• links to: GitHub Pull Request #2479