Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-4718

More-efficient ExecutorService for improved throughput

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.1 rc1
    • None

    Description

      Currently all our execution stages dequeue tasks one at a time. This can result in contention between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue).

      One approach to mitigating this would be to make consumer threads do more work in "bulk" instead of just one task per dequeue. (Producer threads tend to be single-task oriented by nature, so I don't see an equivalent opportunity there.)

      BlockingQueue has a drainTo(collection, int) method that would be perfect for this. However, no ExecutorService in the jdk supports using drainTo, nor could I google one.

      What I would like to do here is create just such a beast and wire it into (at least) the write and read stages. (Other possible candidates for such an optimization, such as the CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.)

      AbstractExecutorService may be useful. The implementations of ICommitLogExecutorService may also be useful. (Despite the name these are not actual ExecutorServices, although they share the most important properties of one.)

      Attachments

        1. v1-stress.out
          11 kB
          Jason Brown
        2. stress op rate with various queues.ods
          37 kB
          Benedict Elliott Smith
        3. stress_2014May16.txt
          10 kB
          Jason Brown
        4. stress_2014May15.txt
          11 kB
          Jason Brown
        5. PerThreadQueue.java
          3 kB
          Vijay
        6. op costs of various queues.ods
          28 kB
          Benedict Elliott Smith
        7. jason_write.svg
          8 kB
          Benedict Elliott Smith
        8. jason_run3.svg
          7 kB
          Benedict Elliott Smith
        9. jason_run2.svg
          7 kB
          Benedict Elliott Smith
        10. jason_run1.svg
          7 kB
          Benedict Elliott Smith
        11. jason_read.svg
          8 kB
          Benedict Elliott Smith
        12. jason_read_latency.svg
          11 kB
          Benedict Elliott Smith
        13. E600M_summary_key_s.svg
          8 kB
          Benedict Elliott Smith
        14. E10M_summary_key_s.svg
          7 kB
          Benedict Elliott Smith
        15. E100M_summary_key_s.svg
          8 kB
          Benedict Elliott Smith
        16. belliotsmith_branches-stress.out.txt
          10 kB
          Jason Brown
        17. baq vs trunk.png
          44 kB
          Ryan McGuire
        18. backpressure-stress.out.txt
          13 kB
          Jason Brown
        19. aws.svg
          9 kB
          Benedict Elliott Smith
        20. aws_read.svg
          8 kB
          Benedict Elliott Smith
        21. austin_diskbound_read.svg
          12 kB
          Benedict Elliott Smith
        22. 4718-v1.patch
          23 kB
          Jason Brown

        Issue Links

          Activity

            People

              benedict Benedict Elliott Smith
              jbellis Jonathan Ellis
              Benedict Elliott Smith
              Jason Brown
              Votes:
              2 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: