Cassandra
  1. Cassandra
  2. CASSANDRA-4924

Make CQL 3 data accessible via thrift.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 3
    • Component/s: Core
    • Labels:

      Description

      Following the changes from CASSANDRA-4377 data created using CQL 3 is not visible via the thrift interface.

      This goes against the spirit of many comments by the project that "the thrift API is not going away". These statements and ones such as "Internally, both CQL3 and thrift use the same storage engine, so all future improvements to this engine will impact both of them equally." (http://www.datastax.com/dev/blog/thrift-to-cql3) and the CQL3 and thrift examples given here http://www.datastax.com/dev/blog/cql3-for-cassandra-experts gave the impression CQL 3 was a layer on top of the core storage engine. It now appears to be an incompatible format change.

      It makes it impossible to explain to existing using users how CQL 3 stores it's data.

      It also creates an all or nothing approach to trying CQL 3.

      My request is to make all data written by CQL 3 readable via the thrift API.

      An example of using the current 1.2 trunk is below:

      cqlsh:cass_college> CREATE TABLE UserTweets 
                      ... (
                      ...     tweet_id    bigint,
                      ...     user_name   text,
                      ...     body        text,
                      ...     timestamp   timestamp,
                      ...     PRIMARY KEY (user_name, tweet_id)
                      ... );
      cqlsh:cass_college> INSERT INTO 
                      ...     UserTweets
                      ...     (tweet_id, body, user_name, timestamp)
                      ... VALUES
                      ...     (1, 'The Tweet', 'fred', 1352150816917);
      cqlsh:cass_college> 
      cqlsh:cass_college> 
      cqlsh:cass_college> select * from UserTweets;
      
       user_name | tweet_id | body      | timestamp
      -----------+----------+-----------+--------------------------
            fred |        1 | The Tweet | 2012-11-06 10:26:56+1300
      

      and in the CLI

      [default@cass_college] show schema;
      create keyspace cass_college
        with placement_strategy = 'SimpleStrategy'
        and strategy_options = {replication_factor : 3}
        and durable_writes = true;
      
      use cass_college;
      
      
      
      [default@cass_college] list UserTweets;
      UserTweets not found in current keyspace.
      [default@cass_college] 
      

        Issue Links

          Activity

          Hide
          Jonathan Ellis added a comment -

          Following the changes from CASSANDRA-4377 data created using CQL 3 is not visible via the thrift interface.

          More accurately, the table definitions are hidden, because you really shouldn't be trying to manually perform CQL3-style encoding. But if you really insist, batch_mutate, get_range_slice, and friends will still work as advertised, and even validate the bytes you give them.

          This seems like a reasonable compromise to me: tools designed for Thrift will not try to scribble over cql3 tables inadvertently because they aren't aware of the difference, but if you know your schema and want to do limited manual encoding that is available to you.

          Show
          Jonathan Ellis added a comment - Following the changes from CASSANDRA-4377 data created using CQL 3 is not visible via the thrift interface. More accurately, the table definitions are hidden, because you really shouldn't be trying to manually perform CQL3-style encoding. But if you really insist, batch_mutate, get_range_slice, and friends will still work as advertised, and even validate the bytes you give them. This seems like a reasonable compromise to me: tools designed for Thrift will not try to scribble over cql3 tables inadvertently because they aren't aware of the difference, but if you know your schema and want to do limited manual encoding that is available to you.
          Hide
          Nate McCall added a comment -

          I can't really "know my schema" programmatically unless describe_keyspace works as advertised (advertised != buried in a readme or similar).

          If we are looking for usability, intentionally adding impedance mismatch between CQL3 and thrift is not helping.

          Show
          Nate McCall added a comment - I can't really "know my schema" programmatically unless describe_keyspace works as advertised (advertised != buried in a readme or similar). If we are looking for usability, intentionally adding impedance mismatch between CQL3 and thrift is not helping.
          Hide
          Jonathan Ellis added a comment -

          You tell me, then. Suppose we add new fields to CfDef for the extra metadata CQL3 uses, and return them in describe_keyspace. How do you keep a tool written for C* 1.0, that doesn't know these fields exists, from misinterpreting it based on the fields that it does know about?

          Show
          Jonathan Ellis added a comment - You tell me, then. Suppose we add new fields to CfDef for the extra metadata CQL3 uses, and return them in describe_keyspace. How do you keep a tool written for C* 1.0, that doesn't know these fields exists, from misinterpreting it based on the fields that it does know about?
          Hide
          amorton added a comment -

          More accurately, the table definitions are hidden, because you really shouldn't be trying to manually perform CQL3-style encoding. But if you really insist, batch_mutate, get_range_slice, and friends will still work as advertised, and even validate the bytes you give them.

          Are you saying this is a restriction in the cli only ?

          This seems like a reasonable compromise to me: tools designed for Thrift will not try to scribble over cql3 tables inadvertently because they aren't aware of the difference, but if you know your schema and want to do limited manual encoding that is available to you.

          Statements like "This CQL definition will store data in exactly the same way than the thrift definition above" (http://www.datastax.com/dev/blog/thrift-to-cql3) Created the impression two API's were using the storage engine in an equal but different way.

          Hiding table definitions from one API because it does things the other will not understand goes against those statements and that impression.

          I'm trying to understand what CQL 3 is, how to explain it, and how to help people migrate to it. It looks like CQL 3 is table orientated, schema driven, API that is incompatible with the previous Thrift/RPC way of using Cassandra.

          That's a big change to come in with an API upgrade.

          IMHO making the CQL 3 data read and/or write visible via thrift would make it easier at a human level. If there is real danger of thrift API clients essentially corrupting CQL 3 data could a config setting be added to allow read and/or write ?

          Show
          amorton added a comment - More accurately, the table definitions are hidden, because you really shouldn't be trying to manually perform CQL3-style encoding. But if you really insist, batch_mutate, get_range_slice, and friends will still work as advertised, and even validate the bytes you give them. Are you saying this is a restriction in the cli only ? This seems like a reasonable compromise to me: tools designed for Thrift will not try to scribble over cql3 tables inadvertently because they aren't aware of the difference, but if you know your schema and want to do limited manual encoding that is available to you. Statements like "This CQL definition will store data in exactly the same way than the thrift definition above" ( http://www.datastax.com/dev/blog/thrift-to-cql3 ) Created the impression two API's were using the storage engine in an equal but different way. Hiding table definitions from one API because it does things the other will not understand goes against those statements and that impression. I'm trying to understand what CQL 3 is, how to explain it, and how to help people migrate to it. It looks like CQL 3 is table orientated, schema driven, API that is incompatible with the previous Thrift/RPC way of using Cassandra. That's a big change to come in with an API upgrade. IMHO making the CQL 3 data read and/or write visible via thrift would make it easier at a human level. If there is real danger of thrift API clients essentially corrupting CQL 3 data could a config setting be added to allow read and/or write ?
          Hide
          Nate McCall added a comment -

          How do you keep a tool written for C* 1.0, that doesn't know these fields exists, from misinterpreting it based on the fields that it does know about?

          Like this is the first change on the CfDef struct? We've been managing that on the client for the past 2.5 years with well typed exceptions (most times when you do something dumb. All version of cassandra and hector 0.8 <-> 1.0 <-> 1.1 work with as much as possible. "Interop works fine - just don't do anything with programatic keyspace/cf stuff" is a frequent reply on our mail list for such. Users are fine with that.

          Show
          Nate McCall added a comment - How do you keep a tool written for C* 1.0, that doesn't know these fields exists, from misinterpreting it based on the fields that it does know about? Like this is the first change on the CfDef struct? We've been managing that on the client for the past 2.5 years with well typed exceptions (most times when you do something dumb. All version of cassandra and hector 0.8 <- > 1.0 < -> 1.1 work with as much as possible. "Interop works fine - just don't do anything with programatic keyspace/cf stuff" is a frequent reply on our mail list for such. Users are fine with that.
          Hide
          Jason Brown added a comment -

          My two cents: The initial impact of trying out the new feature (cql3), and not being to compare it with something I already know (cassandra-cli) is a little confusing. I think Sylvain is onto something when, in the other ticket, he mentions this may be a documentation issue, but where would a new cql3 user (experienced with the old-school way) go to figure out what happened to their shiny new cql3 data?

          While it's a little bit more work of supporting thrift, it might be worth it to ensure a smoother on-ramp for cql3. Perhaps Aaron's idea of a config help lower the barrier to exit from thrift for users.

          Personally, I'd just want the cli, and clients like astyanax/hector, to be able to describe_keyspace to show me the cql3 table exists, and reads would be nice, too, so I could confirm the cql3 data. Anything else is a freebie when trying to on-ramp to cql3.

          Show
          Jason Brown added a comment - My two cents: The initial impact of trying out the new feature (cql3), and not being to compare it with something I already know (cassandra-cli) is a little confusing. I think Sylvain is onto something when, in the other ticket, he mentions this may be a documentation issue, but where would a new cql3 user (experienced with the old-school way) go to figure out what happened to their shiny new cql3 data? While it's a little bit more work of supporting thrift, it might be worth it to ensure a smoother on-ramp for cql3. Perhaps Aaron's idea of a config help lower the barrier to exit from thrift for users. Personally, I'd just want the cli, and clients like astyanax/hector, to be able to describe_keyspace to show me the cql3 table exists, and reads would be nice, too, so I could confirm the cql3 data. Anything else is a freebie when trying to on-ramp to cql3.
          Hide
          Edward Capriolo added a comment -

          We really can not live without it. If your need to insert to this CF from thrift/hector having the CLI output show you the 'old school' way is the most helpful.

          Show
          Edward Capriolo added a comment - We really can not live without it. If your need to insert to this CF from thrift/hector having the CLI output show you the 'old school' way is the most helpful.
          Hide
          Edward Capriolo added a comment -

          I would be equally happy if cqlsh can provide me the information on how to format hector and thrift queries to throw data into this column family.

          Show
          Edward Capriolo added a comment - I would be equally happy if cqlsh can provide me the information on how to format hector and thrift queries to throw data into this column family.
          Hide
          Jonathan Ellis added a comment -

          Statements like "This CQL definition will store data in exactly the same way than the thrift definition above" (http://www.datastax.com/dev/blog/thrift-to-cql3) Created the impression two API's were using the storage engine in an equal but different way. Hiding table definitions from one API because it does things the other will not understand goes against those statements and that impression.

          I'm not 100% sure what you mean by the first part, but the second is easy to elaborate on:

          • CQL3 can always make sense of a Thrift CF definition.
          • The reverse is not true.

          It's the difference between backwards compatibility and forward-compatibility. This is pretty basic.

          We've been managing that on the client for the past 2.5 years with well typed exceptions

          It sounds like you're talking about (say) Hector circa Jan 2013. I'm talking about Hector circa Oct 2012. The latter is not forwards compatible with what we've done in CQL3: if we waved a wand and included all the metadata in CfDef, the already-released versions of Hector can't deal with them and will in fact do the Wrong Thing.

          In other words: previously we've achieved forwards compatibility mostly by luck, because we haven't had to change anything fundamental since 0.7. Now, we have a situation where we can't achieve perfect forwards compatibility, but what we can do is hide the incompatible parts from Hector-2012 so it doesn't shoot itself in the foot.

          Hector-2013 of course can use CQL3 introspection to inspect the schema, and can then proceed to mangle it via Thrift RPC if it really wants to.

          While it's a little bit more work of supporting thrift

          First, I want to repeat that it's less about saving work than it is about salvaging what forwards compatibility we can.

          For the cli specifically, we could indeed show that the table exists (by using the CQL3 api) but then what?

          1. Try to cram it into the same format used by the others? That would lose crucial information. Remember that people use this to "back up" schema, so that would not be a friendly thing to do.
          2. Show the CQL3 definition? That fails the principle of least surprise, the CLI can't actually interpret that syntax. If you want CQL3 syntax use cqlsh.

          We really can not live without it. If your need to insert to this CF from thrift/hector having the CLI output show you the 'old school' way is the most helpful.

          While I still think it's wrongheaded to try to jam CQL3 into an old Hector app, it is reasonable to add CQL3 introspection to the CLI on GET. (SET already works.)

          Pushed this to http://github.com/jbellis/cassandra/branches/4924.

          COMPACT STORAGE works well, because this is "thrift compatibility mode;" PK columns get packed into the comparator, and the last one gets turned into the default validator.

          cqlsh:keyspace1> create table testcompact (a text, b int, c int, primary key (a, b)) with compact storage;
          cqlsh:keyspace1> insert into testcompact (a, b, c) values ('k1', 1, 2);                                
          
          [default@keyspace1] get testcompact['k1'];
          => (column=1, value=2, timestamp=1352474787942000)
          

          "sparse" CQL tables work less well because thrift/cli doesn't really have a way to represent transposed (cql) columns within a (storage engine) row. So all column values show as bytes, but at least you can see them:

          cqlsh:keyspace1> create table testsparse (a text, b int, c int, primary key (a, b));
          cqlsh:keyspace1> insert into testsparse (a, b, c) values ('k1', 1, 2);
          
          [default@keyspace1] get testsparse['k1'];
          => (column=1:, value=, timestamp=1352474739470000)
          => (column=1:c, value=00000002, timestamp=1352474739470000)
          
          Show
          Jonathan Ellis added a comment - Statements like "This CQL definition will store data in exactly the same way than the thrift definition above" ( http://www.datastax.com/dev/blog/thrift-to-cql3 ) Created the impression two API's were using the storage engine in an equal but different way. Hiding table definitions from one API because it does things the other will not understand goes against those statements and that impression. I'm not 100% sure what you mean by the first part, but the second is easy to elaborate on: CQL3 can always make sense of a Thrift CF definition. The reverse is not true. It's the difference between backwards compatibility and forward-compatibility. This is pretty basic. We've been managing that on the client for the past 2.5 years with well typed exceptions It sounds like you're talking about (say) Hector circa Jan 2013. I'm talking about Hector circa Oct 2012. The latter is not forwards compatible with what we've done in CQL3: if we waved a wand and included all the metadata in CfDef, the already-released versions of Hector can't deal with them and will in fact do the Wrong Thing. In other words: previously we've achieved forwards compatibility mostly by luck, because we haven't had to change anything fundamental since 0.7. Now, we have a situation where we can't achieve perfect forwards compatibility, but what we can do is hide the incompatible parts from Hector-2012 so it doesn't shoot itself in the foot. Hector-2013 of course can use CQL3 introspection to inspect the schema, and can then proceed to mangle it via Thrift RPC if it really wants to. While it's a little bit more work of supporting thrift First, I want to repeat that it's less about saving work than it is about salvaging what forwards compatibility we can. For the cli specifically, we could indeed show that the table exists (by using the CQL3 api) but then what? Try to cram it into the same format used by the others? That would lose crucial information. Remember that people use this to "back up" schema, so that would not be a friendly thing to do. Show the CQL3 definition? That fails the principle of least surprise, the CLI can't actually interpret that syntax. If you want CQL3 syntax use cqlsh. We really can not live without it. If your need to insert to this CF from thrift/hector having the CLI output show you the 'old school' way is the most helpful. While I still think it's wrongheaded to try to jam CQL3 into an old Hector app, it is reasonable to add CQL3 introspection to the CLI on GET. (SET already works.) Pushed this to http://github.com/jbellis/cassandra/branches/4924 . COMPACT STORAGE works well, because this is "thrift compatibility mode;" PK columns get packed into the comparator, and the last one gets turned into the default validator. cqlsh:keyspace1> create table testcompact (a text, b int, c int, primary key (a, b)) with compact storage; cqlsh:keyspace1> insert into testcompact (a, b, c) values ('k1', 1, 2); [default@keyspace1] get testcompact['k1']; => (column=1, value=2, timestamp=1352474787942000) "sparse" CQL tables work less well because thrift/cli doesn't really have a way to represent transposed (cql) columns within a (storage engine) row. So all column values show as bytes, but at least you can see them: cqlsh:keyspace1> create table testsparse (a text, b int, c int, primary key (a, b)); cqlsh:keyspace1> insert into testsparse (a, b, c) values ('k1', 1, 2); [default@keyspace1] get testsparse['k1']; => (column=1:, value=, timestamp=1352474739470000) => (column=1:c, value=00000002, timestamp=1352474739470000)
          Hide
          Jason Brown added a comment -

          Hi Jonathan,

          I see your point about not letting 2012 Hector (or astyanax ) do the wrong thing and bork any CQL3 data set it thinks it's safely using. I was thinking skirting around the issue and exposing it thrift was OK, but now, thinking more about it, I'm less convinced of that. Having the CLI be able to have access to CQL3 data, however, I think is the safest route as far as a transition to CQL3.

          I checked out your branch, and was able to see the testcompact table in the cli, but not the testsparse table.

          -Jason

          Show
          Jason Brown added a comment - Hi Jonathan, I see your point about not letting 2012 Hector (or astyanax ) do the wrong thing and bork any CQL3 data set it thinks it's safely using. I was thinking skirting around the issue and exposing it thrift was OK, but now, thinking more about it, I'm less convinced of that. Having the CLI be able to have access to CQL3 data, however, I think is the safest route as far as a transition to CQL3. I checked out your branch, and was able to see the testcompact table in the cli, but not the testsparse table. -Jason
          Hide
          Jonathan Ellis added a comment -

          Odd, can you try with --debug on the cli?

          Show
          Jonathan Ellis added a comment - Odd, can you try with --debug on the cli?
          Hide
          Edward Capriolo added a comment -

          While I still think it's wrongheaded to try to jam CQL3 into an old Hector app, it is reasonable to add CQL3 introspection to the CLI on GET. (SET already works.)

          I need to understand how the table is structured internally so I can deal with it. Sure there are docs out there that explain the internals, but asking the user to read the docs and reverse engineer the CQL table to a thrift table is annoying and error prone. I agree that having half the information is bad in ways, but having no information is much worse.

          I am not trying to be wrongheaded, by jamming CQL3 into hector/thrift. In fact I am trying to understand CQL3 more deeply. This learning and transition is not aided by the missing CFDef. Even if semi-complete it is better then nothing. Also we did say we are going to support thrift clients, and not providing the old style schema is not much support. Even if I go for CQL3 100% I may have some legacy apps that may need to use hector thrift for some time.

          Show
          Edward Capriolo added a comment - While I still think it's wrongheaded to try to jam CQL3 into an old Hector app, it is reasonable to add CQL3 introspection to the CLI on GET. (SET already works.) I need to understand how the table is structured internally so I can deal with it. Sure there are docs out there that explain the internals, but asking the user to read the docs and reverse engineer the CQL table to a thrift table is annoying and error prone. I agree that having half the information is bad in ways, but having no information is much worse. I am not trying to be wrongheaded, by jamming CQL3 into hector/thrift. In fact I am trying to understand CQL3 more deeply. This learning and transition is not aided by the missing CFDef. Even if semi-complete it is better then nothing. Also we did say we are going to support thrift clients, and not providing the old style schema is not much support. Even if I go for CQL3 100% I may have some legacy apps that may need to use hector thrift for some time.
          Hide
          Edward Capriolo added a comment -

          I see your point about not letting 2012 Hector (or astyanax ) do the wrong thing and bork any CQL3 data set it thinks it's safely using. I was thinking skirting around the issue and exposing it thrift was OK, but now, thinking more about it, I'm less convinced of that. Having the CLI be able to have access to CQL3 data, however, I think is the safest route as far as a transition to CQL3.

          http://www.datastax.com/dev/blog/thrift-to-cql3

          But the thrift API is not going anywhere. Existing applications do not have to upgrade to CQL3.

          We don't have to upgrade, but without some schema hints it becomes increasingly more difficult to work with. This is more like a meet-me-half-way deal. To use thrift I need some clue as to what the thrift schema looks like. validation exceptions will help me with the rest.

          Show
          Edward Capriolo added a comment - I see your point about not letting 2012 Hector (or astyanax ) do the wrong thing and bork any CQL3 data set it thinks it's safely using. I was thinking skirting around the issue and exposing it thrift was OK, but now, thinking more about it, I'm less convinced of that. Having the CLI be able to have access to CQL3 data, however, I think is the safest route as far as a transition to CQL3. http://www.datastax.com/dev/blog/thrift-to-cql3 But the thrift API is not going anywhere. Existing applications do not have to upgrade to CQL3. We don't have to upgrade, but without some schema hints it becomes increasingly more difficult to work with. This is more like a meet-me-half-way deal. To use thrift I need some clue as to what the thrift schema looks like. validation exceptions will help me with the rest.
          Hide
          Jason Brown added a comment -

          Edward Capriolo Yes, I read that last week. By 'safest route' I meant how can we see what this new thing is doing so I can (more or less) compare it with my expectations about the old thing. I think once experienced c* folks get the basic hang of the differences from thrift/CF model to CQL3, the need for the intermediate 'cql3 to thrift' conversation may wane quickly. Note: I said 'may'; I'm still thinking about this.

          If I can slightly reframe what Jonathan said, I think it's that existing apps would only need to upgrade when they want to use CQL3 tables (or when thrift is finally retired). Apps that do want to take advantage of CQL3 tables would need to updated, whether that's getting a client that supports cql3 syntax or using the updated protocol. I agree there's a rub when there's a mixed mode of thrift-style CFs and cql3 tables in a keyspace or cluster, but I wonder if that's a documentation issue, as Sylvain alluded to earlier. Of course, how best to communicate that is up for grabs.

          Show
          Jason Brown added a comment - Edward Capriolo Yes, I read that last week. By 'safest route' I meant how can we see what this new thing is doing so I can (more or less) compare it with my expectations about the old thing. I think once experienced c* folks get the basic hang of the differences from thrift/CF model to CQL3, the need for the intermediate 'cql3 to thrift' conversation may wane quickly. Note: I said 'may'; I'm still thinking about this. If I can slightly reframe what Jonathan said, I think it's that existing apps would only need to upgrade when they want to use CQL3 tables (or when thrift is finally retired). Apps that do want to take advantage of CQL3 tables would need to updated, whether that's getting a client that supports cql3 syntax or using the updated protocol. I agree there's a rub when there's a mixed mode of thrift-style CFs and cql3 tables in a keyspace or cluster, but I wonder if that's a documentation issue, as Sylvain alluded to earlier. Of course, how best to communicate that is up for grabs.
          Hide
          Jason Brown added a comment -

          Jonathan EllisTried again with --debug, but no additional output.

          lgmac-jasbrown:jbellis-cassandra $ bin/cassandra-cli --debug
          Connected to: "Test Cluster" on 127.0.0.1/9160
          Welcome to Cassandra CLI version 1.2.0-beta1-SNAPSHOT
          
          Type 'help;' or '?' for help.
          Type 'quit;' or 'exit;' to quit.
          
          [default@unknown] use jeb;
          Authenticated to keyspace: jeb
          [default@jeb] get testsparse['k1'];
          testsparse not found in current keyspace.
          

          The "xyz not found in current keyspace" gets printed from CliCompiler.getColumnFamily() - will poke around a little more.

          Show
          Jason Brown added a comment - Jonathan Ellis Tried again with --debug, but no additional output. lgmac-jasbrown:jbellis-cassandra $ bin/cassandra-cli --debug Connected to: "Test Cluster" on 127.0.0.1/9160 Welcome to Cassandra CLI version 1.2.0-beta1-SNAPSHOT Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit. [ default @unknown] use jeb; Authenticated to keyspace: jeb [ default @jeb] get testsparse['k1']; testsparse not found in current keyspace. The "xyz not found in current keyspace" gets printed from CliCompiler.getColumnFamily() - will poke around a little more.
          Hide
          Jason Brown added a comment - - edited

          FWIW, CliSessionState.debug is only referenced in CliClient.executeUseKeySpace() when a TException is caught .

          Show
          Jason Brown added a comment - - edited FWIW, CliSessionState.debug is only referenced in CliClient.executeUseKeySpace() when a TException is caught .
          Hide
          Jonathan Ellis added a comment -

          Are you sure you did a clean build with my branch? I added code to use cql to grab the extra CF definitions and merge them into what gets passed to the compiler:

              private Iterable<CfDef> currentCfDefs()
              {
                  return Iterables.concat(keyspacesMap.get(keySpace).cf_defs, cql3KeyspacesMap.get(keySpace).values());
              }
          
          Show
          Jonathan Ellis added a comment - Are you sure you did a clean build with my branch? I added code to use cql to grab the extra CF definitions and merge them into what gets passed to the compiler: private Iterable<CfDef> currentCfDefs() { return Iterables.concat(keyspacesMap.get(keySpace).cf_defs, cql3KeyspacesMap.get(keySpace).values()); }
          Hide
          Jason Brown added a comment -

          Crap ... just realized I have my ~/.cassandra.in.sh baked to point to my usual dev space. Lemme try that again

          Show
          Jason Brown added a comment - Crap ... just realized I have my ~/.cassandra.in.sh baked to point to my usual dev space. Lemme try that again
          Hide
          Aleksey Yeschenko added a comment -

          I can confirm that CLI get works on Jonathan's branch:

          [default@keyspace1] get testcompact['k1'];
          => (column=1, value=2, timestamp=1352502050183000)
          Returned 1 results.
          Elapsed time: 19 msec(s).
          [default@keyspace1] get testsparse['k1'];
          => (column=1:, value=, timestamp=1352502075136000)
          => (column=1:c, value=00000002, timestamp=1352502075136000)
          Returned 2 results.
          Elapsed time: 13 msec(s).

          Show
          Aleksey Yeschenko added a comment - I can confirm that CLI get works on Jonathan's branch: [default@keyspace1] get testcompact ['k1'] ; => (column=1, value=2, timestamp=1352502050183000) Returned 1 results. Elapsed time: 19 msec(s). [default@keyspace1] get testsparse ['k1'] ; => (column=1:, value=, timestamp=1352502075136000) => (column=1:c, value=00000002, timestamp=1352502075136000) Returned 2 results. Elapsed time: 13 msec(s).
          Hide
          Jason Brown added a comment -

          Yup, now works for me, too (stoopid computer).

          Show
          Jason Brown added a comment - Yup, now works for me, too (stoopid computer).
          Hide
          Aleksey Yeschenko added a comment -

          +1

          Show
          Aleksey Yeschenko added a comment - +1
          Hide
          Jason Brown added a comment -

          LGTM, +1

          Show
          Jason Brown added a comment - LGTM, +1
          Hide
          Jonathan Ellis added a comment -

          I need to understand how the table is structured internally so I can deal with it. Sure there are docs out there that explain the internals, but asking the user to read the docs and reverse engineer the CQL table to a thrift table is annoying and error prone.

          My point is this: Why would you introduce a cql3 table, if you are planning to use it from Thrift? Don't! Keep developing your Thrift-based app with Thrift-based CF definitions. Or, upgrade your Thrift app to CQL3 first, then start adding CQL3 [non-COMPACT] tables. And when you start a new app, use CQL3 from the beginning and it's a non-issue.

          In any case, I am willing to meet you half way as you say and add this to the CLI. Committed.

          Show
          Jonathan Ellis added a comment - I need to understand how the table is structured internally so I can deal with it. Sure there are docs out there that explain the internals, but asking the user to read the docs and reverse engineer the CQL table to a thrift table is annoying and error prone. My point is this: Why would you introduce a cql3 table, if you are planning to use it from Thrift? Don't! Keep developing your Thrift-based app with Thrift-based CF definitions. Or, upgrade your Thrift app to CQL3 first, then start adding CQL3 [non-COMPACT] tables. And when you start a new app, use CQL3 from the beginning and it's a non-issue. In any case, I am willing to meet you half way as you say and add this to the CLI. Committed.
          Hide
          amorton added a comment -

          It's the difference between backwards compatibility and forward-compatibility. This is pretty basic.

          My comment was not about the difference between backwards and forwards compatibility, or the ease at which it can be understood.

          My point was that up until CASSANDRA-4377 very few people would have thought that, as a new API, CQL stored data that the old API could not see. I certainly cannot remember reading anything about it. The impression I had was that CQL and Thrift were sharing the storage engine.

          Enabling it for COMPACT_STORAGE is a good comprise, thanks for adding it. I think it will give people some training wheels when it come to using CQL3.

          Show
          amorton added a comment - It's the difference between backwards compatibility and forward-compatibility. This is pretty basic. My comment was not about the difference between backwards and forwards compatibility, or the ease at which it can be understood. My point was that up until CASSANDRA-4377 very few people would have thought that, as a new API, CQL stored data that the old API could not see. I certainly cannot remember reading anything about it. The impression I had was that CQL and Thrift were sharing the storage engine. Enabling it for COMPACT_STORAGE is a good comprise, thanks for adding it. I think it will give people some training wheels when it come to using CQL3.
          Hide
          Jonathan Ellis added a comment -

          To clarify: this works for non-COMPACT STORAGE as well ("testsparse" above), with the limitation that I can only give you back cell values as bytes.

          Show
          Jonathan Ellis added a comment - To clarify: this works for non-COMPACT STORAGE as well ("testsparse" above), with the limitation that I can only give you back cell values as bytes.

            People

            • Assignee:
              Jonathan Ellis
              Reporter:
              amorton
              Reviewer:
              Jason Brown
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development