Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9754

Make index info heap friendly for large CQL partitions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Awaiting Feedback
    • Low
    • Resolution: Unresolved
    • 5.x
    • Legacy/Core
    • None

    Description

      Looking at a heap dump of 2.0 cluster, I found that majority of the objects are IndexInfo and its ByteBuffers. This is specially bad in endpoints with large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for GC. Can this be improved by not creating so many objects?

      Attachments

        1. 0f8e28c220fd5af6c7b5dd2d3dab6936c4aa4b6b.patch
          5 kB
          Jeff Jirsa
        2. gc_collection_times_with_birch.png
          172 kB
          Michael Kjellman
        3. gc_collection_times_without_birch.png
          260 kB
          Michael Kjellman
        4. gc_counts_with_birch.png
          114 kB
          Michael Kjellman
        5. gc_counts_without_birch.png
          107 kB
          Michael Kjellman
        6. perf_cluster_1_with_birch_read_latency_and_counts.png
          78 kB
          Michael Kjellman
        7. perf_cluster_1_with_birch_write_latency_and_counts.png
          77 kB
          Michael Kjellman
        8. perf_cluster_2_with_birch_read_latency_and_counts.png
          83 kB
          Michael Kjellman
        9. perf_cluster_2_with_birch_write_latency_and_counts.png
          80 kB
          Michael Kjellman
        10. perf_cluster_3_without_birch_read_latency_and_counts.png
          99 kB
          Michael Kjellman
        11. perf_cluster_3_without_birch_write_latency_and_counts.png
          109 kB
          Michael Kjellman

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-8931 IndexSummary (and Index) should store the token, and the minimal key to unambiguously direct a query

          • Normal -
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-11206 Support large partitions on the 3.0 sstable format

          • Normal -
          • Resolved
          relates to

          Sub-task - The sub-task of the issue CASSANDRA-9738 Migrate key-cache to be fully off-heap

          • Normal -
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. CASSANDRA-9843 Augment or replace partition index with adaptive range filters

          • Normal -
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-9231 Support Routing Key as part of Partition Key

          • Normal -
          • Resolved
          supercedes

          Bug - A problem which impairs or prevents the functions of the product. CASSANDRA-13973 IllegalArgumentException in upgradesstables compaction

          • Normal -
          • In Progress

          Activity

            People

              mkjellman Michael Kjellman
              kohlisankalp Sankalp Kohli
              Michael Kjellman
              Branimir Lambov
              Votes:
              12 Vote for this issue
              Watchers:
              51 Start watching this issue

              Dates

                Created:
                Updated: