Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-5898

nodetool cleanup interacting badly with LCS

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Won't Fix
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
    • Severity:
      Normal

      Description

      Then run cassandra-stress read, cassandra-stress write and "nodetool cleanup" all at the same time, a few times, it can cause overlapping row errors.

      ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-245-Data.db') [DecoratedKey(204853724659241194183955214890519, 30303132343830), DecoratedKey(69227335985728660912035125310473966323, 30393537373332)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-243-Data.db') [DecoratedKey(217896711032704014921095870827202, 30333635363932), DecoratedKey(71430242198281555888954138354238066233, 30333035343132)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.  If you have not yet run scrub, you should do so since you may also have rows out-of-order within an sstable
      ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-213-Data.db') [DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316, 30393733303134)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-247-Data.db') [DecoratedKey(71430378028108400200608251408083114019, 30383133313937), DecoratedKey(142788557025615975974639291059795386925, 30353437373934)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.  If you have not yet run scrub, you should do so since you may also have rows out-of-order within an sstable
      ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,175 LeveledManifest.java (line 223) At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-249-Data.db') [DecoratedKey(138495280291659088173025258556445559667, 30313039353935), DecoratedKey(170139918483433930784261689358244392849, 30313336303439)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-252-Data.db') [DecoratedKey(142788613459989682219570222310727294191, 30383031333839), DecoratedKey(170140890916069420209456014750832266661, 30333731303535)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.  If you have not yet run scrub, you should do so since you may also have rows out-of-order within an sstable
      ERROR [CompactionExecutor:3] 2013-08-16 19:22:28,912 LeveledManifest.java (line 223) At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-271-Data.db') [DecoratedKey(68545052253582928820015773344685671746, 30303739313938), DecoratedKey(119320297902762425433209003370359962009, 30373137323238)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-270-Data.db') [DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316, 30393733303134)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.  If you have not yet run scrub, you should do so since you may also have rows out-of-order within an sstable
      

      Steps to repro:

      1. Run cassandra-stress -Z LeveledCompactionStrategy
      2. Run nodetool cleanup
      3. While cleanup is still running, run cassandra-stress -o write and cassandra-stress -o read at the same time.

        Attachments

        1. system.log
          142 kB
          Jeremiah Jordan

          Activity

            People

            • Assignee:
              yukim Yuki Morishita
              Reporter:
              jjordan Jeremiah Jordan
              Authors:
              Yuki Morishita
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: