[HDDS-3692] Consider avoiding stream/map/sum in write hotpath

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Ozone Filesystem
• Labels: Triaged performance pull-request-available
• Target Version/s: 1.0.0
• Assignee: Rajesh Balamohan
• Reporter: Rajesh Balamohan
• Created: 01/Jun/20 05:39
• Updated: 02/Jun/20 04:43
• Resolved: 02/Jun/20 03:35

Description

https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java#L113

 

public long computeBufferData() {
 return bufferList.stream().mapToInt(ChunkBuffer::position)
 .sum();
 }
  

https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L256

https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java#L264

Getting back to regular iteration should fix the issue. This causes 3x CPU expensive when compared to HDFS flow.

 

 

 

Issue Links

• links to: GitHub Pull Request #1003