1. Cassandra
  2. CASSANDRA-3432

Avoid large array allocation for compressed chunk offsets


    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.0.2
    • Component/s: None
    • Labels:


      For each compressed file we keep the chunk offsets in memory (a long[]). The size of this array is directly proportional to the sstable file and the chunk_length_kb used, but say for a 64GB sstable, we're talking ~8MB in memory by default.

      Without being absolutely huge, this probably makes the life of the GC harder than necessary for the same reasons than CASSANDRA-2466, and this ticket proposes the same solution, i.e. to break down those big array into smaller ones to ease fragmentation.

      Note that this is only a concern for size tiered compaction. But until leveled compaction is battle tested, the default and we know nobody uses size tiered anymore, it's probably worth making the optimization.


        Sylvain Lebresne created issue -
        Sylvain Lebresne made changes -
        Field Original Value New Value
        Attachment 0001-Break-down-large-long-array.patch [ 12501644 ]
        Jonathan Ellis made changes -
        Reviewer xedin
        Pavel Yaskevich made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12640231 ] patch-available, re-open possible [ 12749068 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12749068 ] reopen-resolved, no closed status, patch-avail, testing [ 12754027 ]
        Aleksey Yeschenko made changes -
        Component/s Core [ 12312978 ]


          • Assignee:
            Sylvain Lebresne
            Sylvain Lebresne
            Pavel Yaskevich
          • Votes:
            0 Vote for this issue
            0 Start watching this issue


            • Created: