[HADOOP-17461] Add thread-level IOStatistics Context

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.3.5
• Component/s: fs, fs/azure, fs/s3
• Labels: pull-request-available
• Assignee: Mehakmeet Singh
• Reporter: Steve Loughran
• Created: 08/Jan/21 17:48
• Updated: 06/Feb/24 11:10
• Resolved: 27/Jul/22 10:23

Description

For effective reporting of the iostatistics of individual worker threads, we need a thread-level context which IO components update.

• this contact needs to be passed in two background thread forming work on behalf of a task.
• IO Components (streams, iterators, filesystems) need to update this context statistics as they perform work
• Without double counting anything.

I imagine a ThreadLocal IOStatisticContext which will be updated in the FileSystem API Calls. This context MUST be passed into the background threads used by a task, so that IO is correctly aggregated.

I don't want streams, listIterators &c to do the updating as there is more risk of double counting. However, we need to see their statistics if we want to know things like "bytes discarded in backwards seeks". And I don't want to be updating a shared context object on every read() call.
If all we want is store IO (HEAD, GET, DELETE, list performance etc) then the FS is sufficient.
If we do want the stream-specific detail, then I propose

• caching the context in the constructor
• updating it only in close() or unbuffer() (as we do from S3AInputStream to S3AInstrumenation)
• excluding those we know the FS already collects.

important

when backporting, please follow with HADOOP-18373

Issue Links

• breaks: HADOOP-18416 ITestS3AIOStatisticsContext failure
• causes: HADOOP-18456 NullPointerException in ObjectListingIterator's constructor
• is depended upon by: HADOOP-18373 IOStatisticsContext tuning
• is depended upon by: HADOOP-18367 S3A prefetching to update IOStatisticsContext
• is depended upon by: MAPREDUCE-7398 Improve TestDFSIO to support different filesystem
• is related to: HADOOP-18373 IOStatisticsContext tuning
• links to: GitHub Pull Request #4352
• links to: GitHub Pull Request #4566
• links to: GitHub Pull Request #4639