Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-11018

Drop column in results in corrupted table or tables state (reversible)

    XMLWordPrintableJSON

    Details

    • Severity:
      Low

      Description

      After dropping a column from a table, that table is no longer accessible from various commands.

      Initial command in cqlsh;

      alter table "sensorUnit" drop "lastCouplingCheckTime";
      

      no errors were reported:

      Subsequently, the following commands fail as follows:

      > nodetool compact
      
      root@marble:/var/log/cassandra# nodetool compact
      error: Unknown column lastCouplingCheckTime in table powermon.sensorUnit
      -- StackTrace --
      java.lang.AssertionError: Unknown column lastCouplingCheckTime in table powermon.sensorUnit
              at org.apache.cassandra.db.LegacyLayout.readLegacyAtom(LegacyLayout.java:964)
              at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$AtomIterator.readAtom(UnfilteredDeserializer.java:520)
              at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$AtomIterator.hasNext(UnfilteredDeserializer.java:503)
              at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.readRow(UnfilteredDeserializer.java:446)
              at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:422)
              at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:289)
              at org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:134)
              at org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:57)
              at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:329)
              at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
              at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:65)
              at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:109)
              at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:100)
              at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:442)
              at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
              at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150)
              at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72)
              at org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
              at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:177)
              at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
              at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
              at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
              at org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:572)
              at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      Also I get the following from cqlsh commands:

      cqlsh:sensorTrack> select * from "sensorUnit";
      Traceback (most recent call last):
        File "/usr/bin/cqlsh.py", line 1258, in perform_simple_statement
          result = future.result()
        File "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py", line 3122, in result
          raise self._final_exception
      ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation failed - received 0 responses and 1 failures" info={'failures': 1, 'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
      

      However, after I readded the table, access to the database was restored.

      alter table "sensorUnit" add "lastCouplingCheckTime"
      

      I was not able to reproduce as subsequent attempts to alter worked properly, but the problem occurred on two tables that were altered at the same time so there may be a need to ensure a drop completes entirely when it is performed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                slebresne Sylvain Lebresne
                Reporter:
                longtimer Jason Kania
                Authors:
                Sylvain Lebresne
                Reviewers:
                Marcus Eriksson
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: