Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-4741

Cassandra stops persisting or flushing data

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Duplicate
    • Fix Version/s: 1.1.6
    • Component/s: None
    • Labels:
      None
    • Environment:

      Linux, Windows
      Cassandra 1.1.4 or 1.1.5

    • Severity:
      Normal

      Description

      Hello,

      we have a Cassandra instance running on a single node with Windows XP 32bit.
      After some time of normal working and some restart, Cassandra seems to stop to persist data into the keyspace without raising any error.
      It's possible to update/insert rows or columns and you can query and see the updated keyspace but if you stop and start Cassandra, you get a keyspace 'from the past', always from the same point in the past.

      We were not able to understand how to trigger this inconsistency in the first place but once you get it, you can't get rid of it if not removing the data directories.
      Here in attachment you can find a complete set of Cassandra data directories:

         cassandra-data/
         |-- commitlog
         |-- data
         `-- saved_caches
      

      with which it's possible test the inconsistent scenario.
      We tried this on another clean Win Xp 32 machine and we could get the same issue from the first run.
      On a Win XP 64 and the same dir set, Cassandra didn't get into this issue, but from the log we saw a difference in the disk access mode:

      • XP/64: DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap" instead of 'standard'
      • XP/32: DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard

      Not sure this is the key factor, nevertheless in another test bed we experimented the same issue even on a WinXP 64 node but we couldn't collect the dataset.

      Here a short cassandra-cli history to show the issue:

      [default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
      Connected to: "NEUTRO Cluster" on 192.168.1.33/9160
      
      [default@unknown] use neutro_keyspace;
      Authenticated to keyspace: NEUTRO_Keyspace
      
      [default@NEUTRO_Keyspace] describe;
      Keyspace: NEUTRO_Keyspace:
        Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
        Durable Writes: true
          Options: [replication_factor:1]
        Column Families:
          ColumnFamily: TaskId
            Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
            Default column value validator: org.apache.cassandra.db.marshal.UTF8Type
            Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
            GC grace seconds: 864000
            Compaction min/max thresholds: 4/32
            Read repair chance: 1.0
            DC Local Read repair chance: 0.0
            Replicate on write: true
            Caching: KEYS_ONLY
            Bloom Filter FP chance: default
            Built indexes: []
            Column Metadata:
              Column Name: taskId
                Validation Class: org.apache.cassandra.db.marshal.LongType
            Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
            Compression Options:
              sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor
      
      [default@NEUTRO_Keyspace] list TaskId;
      Using default limit of 100
      Using default column limit of 100
      -------------------
      RowKey: NE_MASTER_TASK_ID
      => (column=taskId, value=47, timestamp=1349090424494000)
      
      1 Row Returned.
      Elapsed time: 93 msec(s).
      
      [default@NEUTRO_Keyspace] set  taskid['NE_MASTER_TASK_ID']['taskId']='55';
      Value inserted.
      Elapsed time: 31 msec(s).
      
      [default@NEUTRO_Keyspace] list TaskId;
      Using default limit of 100
      Using default column limit of 100
      -------------------
      RowKey: NE_MASTER_TASK_ID
      => (column=taskId, value=55, timestamp=1349108488164000)
      
      1 Row Returned.
      Elapsed time: 31 msec(s).
      

      ...Stop and Start Cassandra...

      [default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
      Connected to: "NEUTRO Cluster" on 192.168.1.33/9160
      
      [default@unknown] use neutro_keyspace;
      Authenticated to keyspace: NEUTRO_Keyspace
      
      [default@NEUTRO_Keyspace] list TaskId;
      Using default limit of 100
      Using default column limit of 100
      -------------------
      RowKey: NE_MASTER_TASK_ID
      => (column=taskId, value=47, timestamp=1349090424494000)
      
      1 Row Returned.
      Elapsed time: 110 msec(s).
      

      Thanks for your attention

        Attachments

        1. cassandra-data.minimal.zip
          25 kB
          Mauri Melato

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mauri.melato Mauri Melato
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: