Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-12706

Exception supposedly after ttl expires and compaction occurs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Urgent
    • Resolution: Fixed
    • Fix Version/s: 3.0.10, 3.10
    • Labels:
      None
    • Environment:

      Linux amd64 Amazon default AMI,
      Java 1.8.0_122 oracle jdk

    • Severity:
      Critical
    • Since Version:

      Description

      After TTL (1 week) expires and compaction occurs. We start seeing:

      WARN [SharedPool-Worker-1] 2016-09-26 07:35:17,420 AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread Thread[SharedPool-Worker-1,5,main]: {}
      java.lang.NullPointerException: null
      at org.apache.cassandra.db.Slices$ArrayBackedSlices$ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter.toCQLString(ClusteringIndexSliceFilter.java:150) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.SinglePartitionReadCommand.appendCQLWhereClause(SinglePartitionReadCommand.java:911) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.transform.BasePartitions.runOnClose(BasePartitions.java:70) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_102]
      at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.0.9.jar:3.0.9]
      at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
      WARN [SharedPool-Worker-3] 2016-09-26 07:35:31,247 AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread Thread[SharedPool-Worker-3,5,main]: {}
      java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2470) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_102]
      at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.0.9.jar:3.0.9]
      at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
      Caused by: java.lang.NullPointerException: null
      at org.apache.cassandra.db.Slices$ArrayBackedSlices$ComponentOfSlice.isEQ(Slices.java:748) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.Slices$ArrayBackedSlices.toCQLString(Slices.java:659) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.filter.ClusteringIndexSliceFilter.toCQLString(ClusteringIndexSliceFilter.java:150) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.SinglePartitionReadCommand.appendCQLWhereClause(SinglePartitionReadCommand.java:911) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.ReadCommand.toCQLString(ReadCommand.java:560) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.ReadCommand$1MetricRecording.onClose(ReadCommand.java:506) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.transform.BasePartitions.runOnClose(BasePartitions.java:70) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:76) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1797) ~[apache-cassandra-3.0.9.jar:3.0.9]
      at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2466) ~[apache-cassandra-3.0.9.jar:3.0.9]
      ... 5 common frames omitted

      This results in queries failing on those tables (not always though, at first). The failures keep increasing, until total failure after a few days.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stefania Stefania Alborghetti
                Reporter:
                nikhil_sharma Nikhil Sharma
                Authors:
                Stefania Alborghetti
                Reviewers:
                Swen Fuhrmann
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: