Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8559

OOM caused by large tombstone warning.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Fix Version/s: 2.0.15, 2.1.5
    • Component/s: None
    • Labels:
    • Environment:

      2.0.11 / 2.1

    • Severity:
      Normal
    • Since Version:

      Description

      When running with high amount of tombstones the error message generation from CASSANDRA-6117 can lead to out of memory situation with the default setting.

      Attached a heapdump viewed in visualvm showing how this construct created two 777mb strings to print the error message for a read query and then crashed OOM.

              if (respectTombstoneThresholds() && columnCounter.ignored() > DatabaseDescriptor.getTombstoneWarnThreshold())
              {
                  StringBuilder sb = new StringBuilder();
                  CellNameType type = container.metadata().comparator;
                  for (ColumnSlice sl : slices)
                  {
                      assert sl != null;
      
                      sb.append('[');
                      sb.append(type.getString(sl.start));
                      sb.append('-');
                      sb.append(type.getString(sl.finish));
                      sb.append(']');
                  }
      
                  logger.warn("Read {} live and {} tombstoned cells in {}.{} (see tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={}",
                              columnCounter.live(), columnCounter.ignored(), container.metadata().ksName, container.metadata().cfName, count, sb, container.deletionInfo());
              }
      

        Attachments

        1. stacktrace.log
          3 kB
          Dominic Letz
        2. Selection_048.png
          103 kB
          Dominic Letz
        3. cassandra-2.0.11-8559.txt
          5 kB
          Dominic Letz
        4. 8559.txt
          1 kB
          Aleksey Yeschenko

          Issue Links

            Activity

              People

              • Assignee:
                aleksey Aleksey Yeschenko
                Reporter:
                dominicletz Dominic Letz
                Authors:
                Aleksey Yeschenko
                Reviewers:
                Sam Tunnicliffe
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: