[HDDS-2309] Optimise OzoneManagerDoubleBuffer::flushTransactions to flush in batches

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Manager
• Labels: TriagePending performance
• Target Version/s: 1.5.0
• Assignee: Ritesh Shukla
• Reporter: Rajesh Balamohan
• Created: 15/Oct/19 10:55
• Updated: 16/Dec/23 23:43

Description

When running a write heavy benchmark, org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.flushTransactions was invoked for pretty much every write.

This forces cleanupCache to be invoked which ends up choking in single thread executor. Attaching the profiler information which gives more details.

Ideally, flushTransactions should batch up the work to reduce load on rocksDB.

 

https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L130

 

https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L322

 

 

Issue Links

• is related to: HDDS-3615 Call cleanup on tables only when double buffer has transactions related to tables.