Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16318

Memtable heap size is severely underestimated

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 4.0-rc1, 4.0
    • Local/Memtable
    • None

    Description

      We seem to be estimating the size of the on-heap memtable metadata to be around half of what it actually is. For example, during a read benchmark which writes 1 million single-long rows the memtable reports

      1000000 ops, 58.174MiB serialized bytes, 385.284MiB (19%) on heap, 0.000KiB (0%) off-heap
      

      while a heap dump taken at this point:

      lists an usage of about 666MB altogether.

      Switching to offheap_objects, the reported numbers are

      1000000 ops, 58.174MiB serialized bytes, 233.650MiB (12%) on heap, 53.406MiB (3%) off-heap
      

      while actual heap usage:

      is about 442MB.

      Looking at the code we definitely are not counting the AtomicBTreePartition.Holder, EncodingStats, liveness and deletion info objects associated with each partition, and most probably others.

      Attachments

        1. image-2020-12-09-10-57-21-994.png
          48 kB
          Branimir Lambov
        2. image-2020-12-09-11-01-31-273.png
          37 kB
          Branimir Lambov

        Activity

          People

            e.dimitrova Ekaterina Dimitrova
            blambov Branimir Lambov
            Branimir Lambov, Ekaterina Dimitrova
            Benjamin Lerer, Branimir Lambov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: