Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15810

Default StringTableSize parameter causes GC slowdown

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Triage Needed
    • Priority: Normal
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Local/Config
    • Labels:
      None
    • Change Category:
      Performance
    • Complexity:
      Low Hanging Fruit
    • Platform:
      All
    • Impacts:
      None

      Description

      While looking at tail latency on a Cassandra cluster, it came up that the default StringTableSize in Cassandra is set to a million:

      # Larger interned string table, for gossip's benefit (CASSANDRA-6410)
      -XX:StringTableSize=1000003

      This was done for CASSANDRA-6410 by Jonathan Ellis in '13, to optimize heap usage on a test case, running with 500 nodes and num_tokens=512.

      Until Java 13, this string table is implemented as native code, and has to be traversed entirely during the GC initial marking phase, which is a STW event.

      Some testing on my end shows that the pause time of a GC cycle can be reduced by approximately 10 milliseconds if we lower the string table size back to the Java 8 default of 60013 entries.

      Thus, I would recommend this patch (3.11 branch, similar patch for 4.0):

      diff --git a/conf/jvm.options b/conf/jvm.options
      index 01bb1685b3..c184d18c5d 100644
      --- a/conf/jvm.options
      +++ b/conf/jvm.options
      @@ -107,9 +107,6 @@
       # Per-thread stack size.
       -Xss256k
      
      -# Larger interned string table, for gossip's benefit (CASSANDRA-6410)
      --XX:StringTableSize=1000003
      -
       # Make sure all memory is faulted and zeroed on startup.
       # This helps prevent soft faults in containers and makes
       # transparent hugepage allocation more effective.
       

      It does need some testing on more extreme clusters than I have access to, but I ran some Cassandra nodes with -XX:+PrintStringTableStatistics which suggested that the Java default will suffice here.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tvdw Tom van der Woerdt
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: