Cassandra
  1. Cassandra
  2. CASSANDRA-3432

Avoid large array allocation for compressed chunk offsets

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.0.2
    • Component/s: Core
    • Labels:

      Description

      For each compressed file we keep the chunk offsets in memory (a long[]). The size of this array is directly proportional to the sstable file and the chunk_length_kb used, but say for a 64GB sstable, we're talking ~8MB in memory by default.

      Without being absolutely huge, this probably makes the life of the GC harder than necessary for the same reasons than CASSANDRA-2466, and this ticket proposes the same solution, i.e. to break down those big array into smaller ones to ease fragmentation.

      Note that this is only a concern for size tiered compaction. But until leveled compaction is battle tested, the default and we know nobody uses size tiered anymore, it's probably worth making the optimization.

        Activity

        Sylvain Lebresne created issue -
        Sylvain Lebresne made changes -
        Field Original Value New Value
        Attachment 0001-Break-down-large-long-array.patch [ 12501644 ]
        Jonathan Ellis made changes -
        Reviewer xedin
        Pavel Yaskevich made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12640231 ] patch-available, re-open possible [ 12749068 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12749068 ] reopen-resolved, no closed status, patch-avail, testing [ 12754027 ]

          People

          • Assignee:
            Sylvain Lebresne
            Reporter:
            Sylvain Lebresne
            Reviewer:
            Pavel Yaskevich
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development