[HADOOP-18189] S3PrefetchingInputStream to support status probes when closed

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0, 3.3.6
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.6
• Hadoop Flags: Reviewed
• Assignee: Viraj Jasani Assign to me
• Reporter: Steve Loughran
• Created: 29/Mar/22 15:58
• Updated: 28/Jan/24 08:28
• Resolved: 19/Oct/22 13:38

Description

S3PrefetchingInputStream is a bit over aggressive on raising exceptions/downgrading responses after a stream is closed

• MUST: getPos() to return last read location, or a least 0 (maybe we should add this to filesystem spec)
• MUST: getIOStatistics(). critical for collecting stats in processes
• MUST: seekToNewSource() (it's a no op anyway)
• MAY S3AInputStreamStatistics and getStreamStatistics() though that is only in used in testing...ioStatistics have given us a stable stats api. we may want to tag the method @VisibleForTesting to discourage use.

Issue Links

• links to: GitHub Pull Request #5036