Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-2309

Optimise OzoneManagerDoubleBuffer::flushTransactions to flush in batches

    XMLWordPrintableJSON

    Details

    • Target Version/s:

      Description

      When running a write heavy benchmark, org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.flushTransactions was invoked for pretty much every write.

      This forces cleanupCache to be invoked which ends up choking in single thread executor. Attaching the profiler information which gives more details.

      Ideally, flushTransactions should batch up the work to reduce load on rocksDB.

       

      https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L130

       

      https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L322

       

       

        Attachments

        1. Screenshot 2019-10-15 at 4.19.13 PM.png
          553 kB
          Rajesh Balamohan

          Issue Links

            Activity

              People

              • Assignee:
                bharat Bharat Viswanadham
                Reporter:
                rajesh.balamohan Rajesh Balamohan
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: