Cassandra
  1. Cassandra
  2. CASSANDRA-3585

Intermittent exceptions seen in cassandra 1.0.5 during Reads.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Fix Version/s: 0.8.9, 1.0.6
    • Component/s: Core
    • Labels:
      None
    • Environment:

      RHEL 2.6.32-71.el6.x86_64.
      RAM - 28GB
      11 CPUs of 2.6GHz

      Description

      In my test setup I have cassandra db provisioned with cassandra 0.8.7. The setup is of two data centers. I have upgraded the cassandra to the latest version 1.0.5. I'm seeing following exceptions in cassandra logs -

      ERROR [RequestResponseStage:32] 2011-12-06 14:46:08,150 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[RequestResponseStage:32,5,main]
      java.io.IOError: java.io.EOFException
      at org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:71)
      at org.apache.cassandra.service.ReadCallback.response(ReadCallback.java:126)
      at org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:45)
      at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: java.io.EOFException
      at java.io.DataInputStream.readFully(DataInputStream.java:180)
      at org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:100)
      at org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:81)
      at org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:64)
      ... 6 more

      RF is set to DC1:3,DC2:3 and I/m doing the operations with CL=Local_Quorum.
      I have run nodetool scrub on all the nodes in the ring to verify if it solves the issue but it didn't.

      Thanks,
      Shantanu

      1. 3585.txt
        2 kB
        Jonathan Ellis
      2. 3585-v2.txt
        4 kB
        Jonathan Ellis
      3. CassandraLogs.tar.bz2
        9.29 MB
        Shantanu
      4. metap_system.log.zip
        2.71 MB
        Shantanu
      5. metap_system.log.zip
        3.87 MB
        Shantanu

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Shantanu
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development