[HDDS-9377] Call MutableStat.add(long x) method as a runnable task in a separate service executor

• Type: Sub-task
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.0
• Component/s: OM, S3, SCM
• Labels: pull-request-available
• Assignee: Vyacheslav Tutrinov
• Reporter: Vyacheslav Tutrinov
• Created: 03/Oct/23 10:02
• Updated: 13/Dec/23 00:15

Description

We've made an experiment: load 15mlns of 1k files to the EC bucket and read them (3 clients in 50 threads => 150 threads in total). A high metrics related method contention was detected on org.apache.hadoop.metrics2.lib.MutableStat#add(long, long) (as it;s synchronized) - we hung for about a minute (sum) of a 10-minute profiling session

The metrics-related operation results are not required at the time of execution, hence, they can be executed in another thread pool

Issue Links

• links to: GitHub Pull Request #5409