Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-508

Sequential File Improvement + Performance Tests

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: None

      Description

      https://github.com/franz1981/activemq-artemis/tree/artemis-journal-performance/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/mapped

      I've implemented in the package org.apache.activemq.artemis.core.io.mapped a new memory mapped SequentialFile implementation to support fast write/read operations for OSs that cannot use libaio or need RAM access performances on standard Files.

      The implementation is not thread-safe (it is needed?) and needs more buffer checks (or a complex implementation). I'm using the Netty's PlatformDependent class to perform bulk copy without safepoint's poll issues.
      The implementation is simple but it's good as a proof of concept to compare it against the others: i' ve added a coordinated-omission performance test to measure the latency of a directWrite + OS jitter.

      https://github.com/franz1981/activemq-artemis/blob/artemis-journal-performance/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/aio/SequentialFileBench.java

      The write tests show performances typical of memory mapped file: quoting from Peter Lawrey "for burst of up to 10% of the main memory, it can sustain rates of 1 - 3 GB/second written. e.g. A laptop with 8 GB of memory might handle bursts of 800 MB at a rate of 1 GB per second. A server with 64 GB of memory might handle a burst of 6.5 GB at a rate of 3 GB per second".

      I want to improve the quality of the implementation by:
      1- enforcing the original SequenceFile contract
      2- replacing synchronized reads/writes with lock-free versions
      3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue + EventLoop's poller
      4- addressing all the False-Sharing issues around all the AtomicLong instances used
      5- reducing the garbage produced for the fast paths to 0
      [6- expose try methods to allow direct flow control]
      [7- replacing the Semaphore based rate limiter with a lock free one]

      That's are all proposals, what do you think about it?

      Regards

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                clebertsuconic Clebert Suconic
                Reporter:
                nigro.fra@gmail.com Francesco Nigro
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: