Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7546

AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory



    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Fix Version/s: 2.1.1
    • Component/s: None
    • Labels:
    • Severity:


      In order to preserve atomicity, this code attempts to read, clone/update, then CAS the state of the partition.

      Under heavy contention for updating a single partition this can cause some fairly staggering memory growth (the more cores on your machine the worst it gets).

      Whilst many usage patterns don't do highly concurrent updates to the same partition, hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory allocation rates can be seen (especially when the updates being hinted are small updates to different partitions which can happen very fast on their own) - see CASSANDRA-7545

      It would be best to eliminate/reduce/limit the spinning memory allocation whilst not slowing down the very common un-contended case.


        1. 7546.20_2.txt
          6 kB
          graham sanderson
        2. 7546.20_3.txt
          11 kB
          graham sanderson
        3. 7546.20_4.txt
          16 kB
          graham sanderson
        4. 7546.20_5.txt
          16 kB
          graham sanderson
        5. 7546.20_6.txt
          18 kB
          Benedict Elliott Smith
        6. 7546.20_7.txt
          18 kB
          graham sanderson
        7. 7546.20_7b.txt
          20 kB
          graham sanderson
        8. 7546.20_alt.txt
          16 kB
          graham sanderson
        9. 7546.20_async.txt
          22 kB
          graham sanderson
        10. 7546.20.txt
          4 kB
          Benedict Elliott Smith
        11. 7546.21_v1.txt
          12 kB
          graham sanderson
        12. cassandra-2.1-7546.txt
          10 kB
          graham sanderson
        13. cassandra-2.1-7546-v2.txt
          11 kB
          graham sanderson
        14. cassandra-2.1-7546-v3.txt
          12 kB
          graham sanderson
        15. graph2_7546.png
          245 kB
          graham sanderson
        16. graph3_7546.png
          253 kB
          graham sanderson
        17. graph4_7546.png
          232 kB
          graham sanderson
        18. graphs1.png
          300 kB
          graham sanderson
        19. hint_spikes.png
          25 kB
          graham sanderson
        20. suggestion1_21.txt
          26 kB
          graham sanderson
        21. suggestion1.txt
          30 kB
          graham sanderson
        22. young_gen_gc.png
          75 kB
          graham sanderson

          Issue Links



              • Assignee:
                graham.sanderson graham sanderson
                graham sanderson graham sanderson
                graham sanderson
                Yuki Morishita
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Created: