Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10393

LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Cannot Reproduce
    • Fix Version/s: 2.2.x
    • Component/s: None
    • Environment:
    • Severity:
      Normal
    • Since Version:

      Description

      When trying to repair full on a table with the following schema my nodes stall
      and end up with spamming this

      I've recently changed the table from SizeTieredCompactionStrategy to LeveledCompactionStrategy.

      Coming from 2.1.9 -> 2.2.0 -> 2.2.1 i ran upgradesstable without issue as well

      When trying to full repair post compaction change, I got "out of order" errors. A few google searches later, I was told to "scrub" the keyspace - did that during the night (no problems logged, and no data lost)

      Now a repair just stalls and output memory leaks all over the place

      CREATE KEYSPACE sessions WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
      
      CREATE TABLE sessions.sessions (
          id text PRIMARY KEY,
          client_ip text,
          controller text,
          controller_action text,
          created timestamp,
          data text,
          expires timestamp,
          http_host text,
          modified timestamp,
          request_agent text,
          request_agent_bot boolean,
          request_path text,
          site_id int,
          user_id int
      ) WITH bloom_filter_fp_chance = 0.01
          AND caching = '{"keys":"NONE", "rows_per_partition":"NONE"}'
          AND comment = ''
          AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
          AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
          AND dclocal_read_repair_chance = 0.1
          AND default_time_to_live = 0
          AND gc_grace_seconds = 864000
          AND max_index_interval = 2048
          AND memtable_flush_period_in_ms = 0
          AND min_index_interval = 128
          AND read_repair_chance = 0.0
          AND speculative_retry = '99.0PERCENTILE';
      
      ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@4428a373) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
      ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@368dd97) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
      ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@66fb78be) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
      ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@9fdd2e6) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big was not released before the reference was garbage collected
      ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@84fcb91) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big was not released before the reference was garbage collected
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jippignu Christian Winther
            • Votes:
              5 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: