Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-6374

AssertionError for rows with zero columns

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 2.0.3
    • Component/s: None
    • Labels:
      None

      Description

      After upgrading from 1.2.5 to 1.2.9 and then to 2.0.2 we've got those exceptions:

      ERROR [FlushWriter:1] 2013-11-18 16:14:36,305 CassandraDaemon.java (line 187) Exception in thread Thread[FlushWriter:1,5,main]
      java.lang.AssertionError
              at org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:198)
              at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:186)
              at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:360)
              at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:315)
              at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
              at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
              at java.lang.Thread.run(Thread.java:722)
      

      Also found similar issue in this thread:
      http://www.mail-archive.com/user@cassandra.apache.org/msg32875.html
      There Aaron Morton said that its caused by leaving rows with zero columns - that's exactly what we do in some CFs (using Thrift & Astyanax).

      1. 6374.txt
        0.8 kB
        Jonathan Ellis

        Activity

        Hide
        iamaleksey Aleksey Yeschenko added a comment -

        Can this be fixed for 2.0.3 please? We need to upgrade our testing cluster to 2.0.x but cannot because of this issue.

        2.0.3 vote is being restarted, so yes, it will be in 2.0.3.

        Show
        iamaleksey Aleksey Yeschenko added a comment - Can this be fixed for 2.0.3 please? We need to upgrade our testing cluster to 2.0.x but cannot because of this issue. 2.0.3 vote is being restarted, so yes, it will be in 2.0.3.
        Hide
        ash2k Mikhail Mazurskiy added a comment -

        Can this be fixed for 2.0.3 please? We need to upgrade our testing cluster to 2.0.x but cannot because of this issue.

        Show
        ash2k Mikhail Mazurskiy added a comment - Can this be fixed for 2.0.3 please? We need to upgrade our testing cluster to 2.0.x but cannot because of this issue.
        Hide
        jbellis Jonathan Ellis added a comment -

        Committed

        Show
        jbellis Jonathan Ellis added a comment - Committed
        Hide
        jasobrown Jason Brown added a comment -

        +1. Agreed that flushing a zero column row is not a good idea.

        Show
        jasobrown Jason Brown added a comment - +1. Agreed that flushing a zero column row is not a good idea.
        Hide
        jbellis Jonathan Ellis added a comment -

        Patch attached to do that. I'm a bit nervous though that I'm not 100% sure if 1.2 was actually writing empty rows to sstables, or if it was rejecting them silently somewhere else.

        Show
        jbellis Jonathan Ellis added a comment - Patch attached to do that. I'm a bit nervous though that I'm not 100% sure if 1.2 was actually writing empty rows to sstables, or if it was rejecting them silently somewhere else.
        Hide
        jbellis Jonathan Ellis added a comment -

        We still support that. The assertion only rejects no cells at all. (A tombstone still counts as a cell.)

        That said, it's really the memtable's job to not flush empty rows even if it's rather antisocial to give it a batch containing zero mutations.

        Show
        jbellis Jonathan Ellis added a comment - We still support that. The assertion only rejects no cells at all. (A tombstone still counts as a cell.) That said, it's really the memtable's job to not flush empty rows even if it's rather antisocial to give it a batch containing zero mutations.
        Hide
        ash2k Mikhail Mazurskiy added a comment -

        Our scenario of usage is as follows:
        We delete specific columns in a specific row. Sometimes those columns are the last ones. How can we delete the whole row if they are the last ones? We cannot read-check-delete_row_or_only_columns because it has a race between check and delete.

        Show
        ash2k Mikhail Mazurskiy added a comment - Our scenario of usage is as follows: We delete specific columns in a specific row. Sometimes those columns are the last ones. How can we delete the whole row if they are the last ones? We cannot read-check-delete_row_or_only_columns because it has a race between check and delete.
        Hide
        jbellis Jonathan Ellis added a comment -

        Rows that sort-of exist for a non-deterministic amount of time are not a feature I intend to support. That it works in 1.2.x is a bug.

        Show
        jbellis Jonathan Ellis added a comment - Rows that sort-of exist for a non-deterministic amount of time are not a feature I intend to support. That it works in 1.2.x is a bug.
        Hide
        ash2k Mikhail Mazurskiy added a comment -

        Jonathan Ellis can you explain please what exactly is a bad idea here? We are ok with "a row with zero columns will be removed as soon as it compacts". We just didn't expected this usage to be a problem - is it documented somewhere? It works fine with 1.2.x and do not work with 2.0.x.

        Show
        ash2k Mikhail Mazurskiy added a comment - Jonathan Ellis can you explain please what exactly is a bad idea here? We are ok with "a row with zero columns will be removed as soon as it compacts". We just didn't expected this usage to be a problem - is it documented somewhere? It works fine with 1.2.x and do not work with 2.0.x.
        Hide
        jbellis Jonathan Ellis added a comment -

        Under Thrift, a row with zero columns will be removed as soon as it compacts. So this really is a bad idea.

        Show
        jbellis Jonathan Ellis added a comment - Under Thrift, a row with zero columns will be removed as soon as it compacts. So this really is a bad idea.

          People

          • Assignee:
            jbellis Jonathan Ellis
            Reporter:
            sv3k Anton Gorbunov
            Reviewer:
            Jason Brown
          • Votes:
            3 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development