Statements like "This CQL definition will store data in exactly the same way than the thrift definition above" (http://www.datastax.com/dev/blog/thrift-to-cql3) Created the impression two API's were using the storage engine in an equal but different way. Hiding table definitions from one API because it does things the other will not understand goes against those statements and that impression.
I'm not 100% sure what you mean by the first part, but the second is easy to elaborate on:
- CQL3 can always make sense of a Thrift CF definition.
- The reverse is not true.
It's the difference between backwards compatibility and forward-compatibility. This is pretty basic.
We've been managing that on the client for the past 2.5 years with well typed exceptions
It sounds like you're talking about (say) Hector circa Jan 2013. I'm talking about Hector circa Oct 2012. The latter is not forwards compatible with what we've done in CQL3: if we waved a wand and included all the metadata in CfDef, the already-released versions of Hector can't deal with them and will in fact do the Wrong Thing.
In other words: previously we've achieved forwards compatibility mostly by luck, because we haven't had to change anything fundamental since 0.7. Now, we have a situation where we can't achieve perfect forwards compatibility, but what we can do is hide the incompatible parts from Hector-2012 so it doesn't shoot itself in the foot.
Hector-2013 of course can use CQL3 introspection to inspect the schema, and can then proceed to mangle it via Thrift RPC if it really wants to.
While it's a little bit more work of supporting thrift
First, I want to repeat that it's less about saving work than it is about salvaging what forwards compatibility we can.
For the cli specifically, we could indeed show that the table exists (by using the CQL3 api) but then what?
- Try to cram it into the same format used by the others? That would lose crucial information. Remember that people use this to "back up" schema, so that would not be a friendly thing to do.
- Show the CQL3 definition? That fails the principle of least surprise, the CLI can't actually interpret that syntax. If you want CQL3 syntax use cqlsh.
We really can not live without it. If your need to insert to this CF from thrift/hector having the CLI output show you the 'old school' way is the most helpful.
While I still think it's wrongheaded to try to jam CQL3 into an old Hector app, it is reasonable to add CQL3 introspection to the CLI on GET. (SET already works.)
Pushed this to http://github.com/jbellis/cassandra/branches/4924.
COMPACT STORAGE works well, because this is "thrift compatibility mode;" PK columns get packed into the comparator, and the last one gets turned into the default validator.
cqlsh:keyspace1> create table testcompact (a text, b int, c int, primary key (a, b)) with compact storage;
cqlsh:keyspace1> insert into testcompact (a, b, c) values ('k1', 1, 2);
[default@keyspace1] get testcompact['k1'];
=> (column=1, value=2, timestamp=1352474787942000)
"sparse" CQL tables work less well because thrift/cli doesn't really have a way to represent transposed (cql) columns within a (storage engine) row. So all column values show as bytes, but at least you can see them:
cqlsh:keyspace1> create table testsparse (a text, b int, c int, primary key (a, b));
cqlsh:keyspace1> insert into testsparse (a, b, c) values ('k1', 1, 2);
[default@keyspace1] get testsparse['k1'];
=> (column=1:, value=, timestamp=1352474739470000)
=> (column=1:c, value=00000002, timestamp=1352474739470000)