Cassandra
  1. Cassandra
  2. CASSANDRA-2809

In the Cli, update column family <cf> with comparator; create Column metadata

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 0.7.8, 0.8.2
    • Component/s: Tools
    • Labels:
      None
    • Environment:

      Ubuntu 10.10, 32bit
      java version "1.6.0_24"
      installed from Debian packages of Brisk-beta2

      Description

      Using cassandra-cli, I can't update the comparator of a column family with the type I want and when I did it with BytesType, Column metadata appear for each of my existing columns.
      Step to reproduce:

      [default@unknown] create keyspace Test
          with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy'
          and strategy_options = [{replication_factor:1}];
      
      [default@unknown] use Test;
      Authenticated to keyspace: Test
      
      [default@Test] create column family test;
      
      [default@Test] describe keyspace;
      ...
          ColumnFamily: test
            Key Validation Class: org.apache.cassandra.db.marshal.BytesType
            Default column value validator: org.apache.cassandra.db.marshal.BytesType
            Columns sorted by: org.apache.cassandra.db.marshal.BytesType
            Row cache size / save period in seconds: 0.0/0
            Key cache size / save period in seconds: 200000.0/14400
            Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
            GC grace seconds: 864000
            Compaction min/max thresholds: 4/32
            Read repair chance: 1.0
            Replicate on write: false
            Built indexes: []
      ...
      
      [default@Test] update column family test with comparator = 'LongType';
      comparators do not match.
      

      why?? the CF is empty

      [default@Test] update column family test with comparator = 'BytesType';
      f8e4dcb0-9cca-11e0-0000-d0583497e7ff
      Waiting for schema agreement...
      ... schemas agree across the cluster
      
      [default@Test] describe keyspace;
      ...
          ColumnFamily: test
            Key Validation Class: org.apache.cassandra.db.marshal.BytesType
            Default column value validator: org.apache.cassandra.db.marshal.BytesType
            Columns sorted by: org.apache.cassandra.db.marshal.BytesType
            Row cache size / save period in seconds: 0.0/0
            Key cache size / save period in seconds: 200000.0/14400
            Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
            GC grace seconds: 864000
            Compaction min/max thresholds: 4/32
            Read repair chance: 1.0
            Replicate on write: false
            Built indexes: []
      ...
      
      [default@Test] set test[ascii('row1')][long(1)]=integer(35);
      set test[ascii('row1')][long(2)]=integer(36);
      set test[ascii('row1')][long(3)]=integer(38);
      set test[ascii('row2')][long(1)]=integer(45);
      set test[ascii('row2')][long(2)]=integer(42);
      set test[ascii('row2')][long(3)]=integer(33);
      
      [default@Test] list test;
      Using default limit of 100
      -------------------
      RowKey: 726f7731
      => (column=0000000000000001, value=35, timestamp=1308744931122000)
      => (column=0000000000000002, value=36, timestamp=1308744931124000)
      => (column=0000000000000003, value=38, timestamp=1308744931125000)
      -------------------
      RowKey: 726f7732
      => (column=0000000000000001, value=45, timestamp=1308744931127000)
      => (column=0000000000000002, value=42, timestamp=1308744931128000)
      => (column=0000000000000003, value=33, timestamp=1308744932722000)
      
      2 Rows Returned.
      
      [default@Test] update column family test with comparator = 'LongType';
      comparators do not match.
      

      same question than before, my columns contains only long, why I can't?

      [default@Test] update column family test with comparator = 'BytesType';
      
      [default@Test] describe keyspace;                                      
      Keyspace: Test:
        Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
          Options: [replication_factor:1]
        Column Families:
          ColumnFamily: test
            Key Validation Class: org.apache.cassandra.db.marshal.BytesType
            Default column value validator: org.apache.cassandra.db.marshal.BytesType
            Columns sorted by: org.apache.cassandra.db.marshal.BytesType
            Row cache size / save period in seconds: 0.0/0
            Key cache size / save period in seconds: 200000.0/14400
            Memtable thresholds: 0.571875/122/1440 (millions of ops/MB/minutes)
            GC grace seconds: 864000
            Compaction min/max thresholds: 4/32
            Read repair chance: 1.0
            Replicate on write: false
            Built indexes: []
            Column Metadata:
              Column Name:  (0000000000000001)
                Validation Class: org.apache.cassandra.db.marshal.IntegerType
              Column Name:  (0000000000000003)
                Validation Class: org.apache.cassandra.db.marshal.IntegerType
              Column Name:  (0000000000000002)
                Validation Class: org.apache.cassandra.db.marshal.IntegerType
      

      Column Metadata appear from nowhere. I don't think that it's expected.

      1. CASSANDRA-2809.patch
        2 kB
        Pavel Yaskevich
      2. 2809-validate.txt
        0.6 kB
        Jonathan Ellis

        Activity

        Hide
        Jonathan Ellis added a comment -

        Changing comparators is not allowed, since the point of a comparator is that data within a row will be sorted on disk by the comparator's ordering. Changing the comparator without rewriting the data would corrupt the sstable.

        Not sure where that column_metadata comes from though. Looks like a bug.

        Show
        Jonathan Ellis added a comment - Changing comparators is not allowed, since the point of a comparator is that data within a row will be sorted on disk by the comparator's ordering. Changing the comparator without rewriting the data would corrupt the sstable. Not sure where that column_metadata comes from though. Looks like a bug.
        Hide
        Jonathan Ellis added a comment -

        Noticed that update_cf doesn't validate the given CFMetaData. patch to add this. (Does not address the column_metadata problem.)

        Show
        Jonathan Ellis added a comment - Noticed that update_cf doesn't validate the given CFMetaData. patch to add this. (Does not address the column_metadata problem.)
        Hide
        Pavel Yaskevich added a comment -

        That column metadata is a feature: when you use a function call in SET statement it is storing information about validation class for individual columns locally (without sync with server) that was added in context of CASSANDRA-1635.

        +1 of validate patch.

        Show
        Pavel Yaskevich added a comment - That column metadata is a feature: when you use a function call in SET statement it is storing information about validation class for individual columns locally (without sync with server) that was added in context of CASSANDRA-1635 . +1 of validate patch.
        Hide
        Brandon Williams added a comment -

        Committed.

        Show
        Brandon Williams added a comment - Committed.
        Hide
        Jonathan Ellis added a comment -

        That column metadata is a feature: when you use a function call in SET statement it is storing information about validation class for individual columns locally

        Ah, I remember that. But this shouldn't show up in a describe keyspace or we cause the above confusion.

        Show
        Jonathan Ellis added a comment - That column metadata is a feature: when you use a function call in SET statement it is storing information about validation class for individual columns locally Ah, I remember that. But this shouldn't show up in a describe keyspace or we cause the above confusion.
        Hide
        Pavel Yaskevich added a comment - - edited

        How should it be stored on your opinion?

        Show
        Pavel Yaskevich added a comment - - edited How should it be stored on your opinion?
        Hide
        Jonathan Ellis added a comment -

        I'd probably keep a "local" CFMetadata around and check that first.

        Show
        Jonathan Ellis added a comment - I'd probably keep a "local" CFMetadata around and check that first.
        Hide
        Hudson added a comment -

        Integrated in Cassandra-0.8 #216 (See https://builds.apache.org/job/Cassandra-0.8/216/)
        cli validates CFMetaData on update.
        Patch by jbellis, reviewed by Pavel Yaskevich for CASSANDRA-2809

        brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1147261
        Files :

        • /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
        Show
        Hudson added a comment - Integrated in Cassandra-0.8 #216 (See https://builds.apache.org/job/Cassandra-0.8/216/ ) cli validates CFMetaData on update. Patch by jbellis, reviewed by Pavel Yaskevich for CASSANDRA-2809 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1147261 Files : /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java
        Hide
        Pavel Yaskevich added a comment -

        I took a look the code now and what I see is that describe keyspace always calling `thrift.describe_keyspace()` but in the example after adding rows you did a successful "update column family test with comparator = 'BytesType';" which persisted changes in column metadata. I think this is actually a good thing to have we just need to document that, what do you think, Jonathan?

        Show
        Pavel Yaskevich added a comment - I took a look the code now and what I see is that describe keyspace always calling `thrift.describe_keyspace()` but in the example after adding rows you did a successful "update column family test with comparator = 'BytesType';" which persisted changes in column metadata. I think this is actually a good thing to have we just need to document that, what do you think, Jonathan?
        Hide
        Jonathan Ellis added a comment -

        To make sure we're on the same page, here's what I'm talking about:

              Column Metadata:
                Column Name:  (0000000000000001)
                  Validation Class: org.apache.cassandra.db.marshal.IntegerType
                Column Name:  (0000000000000003)
        

        We shouldn't be persisting that or showing it to the user as part of describe keyspace.

        Show
        Jonathan Ellis added a comment - To make sure we're on the same page, here's what I'm talking about: Column Metadata: Column Name: (0000000000000001) Validation Class: org.apache.cassandra.db.marshal.IntegerType Column Name: (0000000000000003) We shouldn't be persisting that or showing it to the user as part of describe keyspace.
        Hide
        Pavel Yaskevich added a comment -

        Ok, I will attach a patch fixing this in a few.

        Show
        Pavel Yaskevich added a comment - Ok, I will attach a patch fixing this in a few.
        Hide
        Pavel Yaskevich added a comment -

        patch for version 0.7 but can be applied on 0.8 also.

        Show
        Pavel Yaskevich added a comment - patch for version 0.7 but can be applied on 0.8 also.
        Hide
        Jonathan Ellis added a comment -

        does this fix the describe problem too, or just the update one?

        Show
        Jonathan Ellis added a comment - does this fix the describe problem too, or just the update one?
        Hide
        Pavel Yaskevich added a comment -

        Describe too.

        Show
        Pavel Yaskevich added a comment - Describe too.
        Hide
        Jonathan Ellis added a comment -

        committed

        Show
        Jonathan Ellis added a comment - committed
        Hide
        Hudson added a comment -

        Integrated in Cassandra-0.7 #530 (See https://builds.apache.org/job/Cassandra-0.7/530/)
        avoid including inferred types in CFupdate
        patch by pyaskevich; reviewed by jbellis for CASSANDRA-2809

        jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1147621
        Files :

        • /cassandra/branches/cassandra-0.7/CHANGES.txt
        • /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java
        Show
        Hudson added a comment - Integrated in Cassandra-0.7 #530 (See https://builds.apache.org/job/Cassandra-0.7/530/ ) avoid including inferred types in CFupdate patch by pyaskevich; reviewed by jbellis for CASSANDRA-2809 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1147621 Files : /cassandra/branches/cassandra-0.7/CHANGES.txt /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/cli/CliClient.java

          People

          • Assignee:
            Pavel Yaskevich
            Reporter:
            Silvère Lestang
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development