Cassandra
  1. Cassandra
  2. CASSANDRA-3237

refactor super column implmentation to use composite column names instead

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 2.0 beta 1
    • Component/s: None
    • Labels:

      Description

      super columns are annoying. composite columns offer a better API and performance. people should use composites over super columns. some people are already using super columns. C* should implement the super column API in terms of composites to reduce code, complexity and testing as well as increase performance.

        Activity

        Hide
        Stu Hood added a comment -

        In order to continue to support the existing API (delete the children of this parent), we'll need to add an internal representation of deleted slices. CASSANDRA-674 can persist them, but ColumnFamily would also need a slice tombstones list.

        Show
        Stu Hood added a comment - In order to continue to support the existing API (delete the children of this parent), we'll need to add an internal representation of deleted slices. CASSANDRA-674 can persist them, but ColumnFamily would also need a slice tombstones list.
        Hide
        Tyler Hobbs added a comment -

        We would also need to support fetching multiple non-contiguous slices from the same row in order to emulate fetching multiple super columns by name.

        Show
        Tyler Hobbs added a comment - We would also need to support fetching multiple non-contiguous slices from the same row in order to emulate fetching multiple super columns by name.
        Hide
        Jeremiah Jordan added a comment -

        Multiple Slice Ranges in one query would be nice for the regular API to get access to if that is implemented.

        Show
        Jeremiah Jordan added a comment - Multiple Slice Ranges in one query would be nice for the regular API to get access to if that is implemented.
        Hide
        Mck SembWever added a comment -

        We using supercolumns to provide us "buckets of counters". (I've attached the irc log discussing our valid use case for supercolumns).

        Today with supercolumns you can select multiple supercolumns in one query.
        With composite columns you have to do a slice for each bucket. In our use case this explodes one query into hundreds of queries.

        Will the existing supercoumn API exist so we can continue doing single queries, or will it become possible to do "get_multislice" (get one row with mutliple column slices)?

        Show
        Mck SembWever added a comment - We using supercolumns to provide us "buckets of counters". (I've attached the irc log discussing our valid use case for supercolumns). Today with supercolumns you can select multiple supercolumns in one query. With composite columns you have to do a slice for each bucket. In our use case this explodes one query into hundreds of queries. Will the existing supercoumn API exist so we can continue doing single queries, or will it become possible to do "get_multislice" (get one row with mutliple column slices)?
        Hide
        Jonathan Ellis added a comment -

        Will the existing supercoumn API exist

        Yes. The first subtask here is about selecting multiple ranges so that we can emulate the "multiple supercolumns in one query" functionality.

        Show
        Jonathan Ellis added a comment - Will the existing supercoumn API exist Yes. The first subtask here is about selecting multiple ranges so that we can emulate the "multiple supercolumns in one query" functionality.
        Hide
        Mck SembWever added a comment -

        Oops, helps to read the issue

        Show
        Mck SembWever added a comment - Oops, helps to read the issue
        Hide
        Robbie Strickland added a comment -

        Will there be a migration path for the underlying SC data? I have two years worth of data in SCs for the same reason as Mck. Having experienced the pain of the 0.6.x -> 0.7.x migration, I'm not thrilled about the idea of figuring out how to get all my data into a new format. No matter how much the C* community may despise SCs, a lot of people have a lot of data stored in them for valid reasons, so there needs to be a well-defined migration path.

        Show
        Robbie Strickland added a comment - Will there be a migration path for the underlying SC data? I have two years worth of data in SCs for the same reason as Mck. Having experienced the pain of the 0.6.x -> 0.7.x migration, I'm not thrilled about the idea of figuring out how to get all my data into a new format. No matter how much the C* community may despise SCs, a lot of people have a lot of data stored in them for valid reasons, so there needs to be a well-defined migration path.
        Hide
        Sylvain Lebresne added a comment -

        @Robbie Don't worry, there will be a migration path or we won't do it. This has never been about breaking this for people, only about cleaning the internals of Cassandra.

        Show
        Sylvain Lebresne added a comment - @Robbie Don't worry, there will be a migration path or we won't do it. This has never been about breaking this for people, only about cleaning the internals of Cassandra.
        Hide
        Vijay added a comment -

        I think the effort will be in 2 phases.

        Phase I:
        Super columns will internally use composite column for storage but will communicate between nodes as Super columns. (for wire compatibility during upgrade)
        We should also modify the startup to scrub the SCF and convert internally to Composite columns.

        Phase II:
        Make internode communication to use columns instead, Cleanup supercolumn specific code in the next major release.

        What this means that the user has to upgrade to 1.3 before upgrading to 1.4 (if he/she uses SC).

        Show
        Vijay added a comment - I think the effort will be in 2 phases. Phase I: Super columns will internally use composite column for storage but will communicate between nodes as Super columns. (for wire compatibility during upgrade) We should also modify the startup to scrub the SCF and convert internally to Composite columns. Phase II: Make internode communication to use columns instead, Cleanup supercolumn specific code in the next major release. What this means that the user has to upgrade to 1.3 before upgrading to 1.4 (if he/she uses SC).
        Hide
        Sylvain Lebresne added a comment -

        I'm not really sure we need to really have 2 phases, and in any case I'm not convinced a startup scrub is the right approach.

        I think that what we need is to write conversions functions for <CF with SC> to and from <CF with equivalent composite> and for requests on both kind of CF. With those, I think you can directly remove the use of SC internally, you just use those functions for 1) compatibility when sending/receiving to/from older nodes and 2) decode old SC format when reading old sstables. Of course that's much more easily said than done.

        Show
        Sylvain Lebresne added a comment - I'm not really sure we need to really have 2 phases, and in any case I'm not convinced a startup scrub is the right approach. I think that what we need is to write conversions functions for <CF with SC> to and from <CF with equivalent composite> and for requests on both kind of CF. With those, I think you can directly remove the use of SC internally, you just use those functions for 1) compatibility when sending/receiving to/from older nodes and 2) decode old SC format when reading old sstables. Of course that's much more easily said than done.
        Hide
        Rustam Aliyev added a comment -

        Is there any page/post describing how exactly SC will be implemented using Composite Columns?

        Show
        Rustam Aliyev added a comment - Is there any page/post describing how exactly SC will be implemented using Composite Columns?
        Hide
        Sylvain Lebresne added a comment -

        There isn't, but it's not like there is many way to do it. It'll be a composite with 2 components, the first one being the old SC name, and the second one being the old column name.

        Show
        Sylvain Lebresne added a comment - There isn't, but it's not like there is many way to do it. It'll be a composite with 2 components, the first one being the old SC name, and the second one being the old column name.
        Hide
        Mck SembWever added a comment - - edited

        ... in any case I'm not convinced a startup scrub is the right approach. I think that what we need is to write conversions functions...

        Despite there being no startup scrub this still means that a manual `nodetool upgradesstables` will use these conversions functions to rewrite all sstables to composite columns?

        Show
        Mck SembWever added a comment - - edited ... in any case I'm not convinced a startup scrub is the right approach. I think that what we need is to write conversions functions... Despite there being no startup scrub this still means that a manual `nodetool upgradesstables` will use these conversions functions to rewrite all sstables to composite columns?
        Hide
        Sylvain Lebresne added a comment -

        Attached patches for this at https://github.com/pcmanus/cassandra/commits/3237-1.

        This ain't small so I'll try to explain the main idea here.

        The main idea is that internally, super column families are handled for almost all intents and purposes as if their comparator was a simple CompositeType with 2 components: the 1st one is the old super column name, the 2nd one the old sub-column name. Meaning that they are largely not a special anymore and all the super column specific code go away (including SuperColumn.java).

        Now for compatibility sake, the main action is in the new SuperColumns.java class. This class contains a bunch of static methods that:

        • deserialize old super column format directly into new composite based CF.
        • serialize new composite based CF to the old super column format
        • convert 'super column query filters' to and from 'composite based query filters'.

        Then in ColumnFamilySerializer and the ReadCommand serializer, we use those static methods when talking to old nodes (and a super column family is involved). We also convert thrift SC queries into equivalent ones on the new composite format in CassandraServer.java.

        The patch also don't shy away from removing abstractions that are not necessary anymore once super columns are removed. Most notably:

        • QueryPath is removed. It was honestly already kind of useless with super columns but even more so without them. It was also error-prone imho because some method that were taking a QueryPath were actually ignoring everything except the columnFamilyName for instance. I note that the class itself is not removed but kept only to simplify wire compatibility with old nodes.
        • IColumn and IColumnContainer are removed.

        We could also merge ColumnFamily and AbstractColumnContainer but I've left that to later.

        As far as testing goes:

        • the unit tests pass more or less. There's CassandraServerTest that timeout on my box, but it does so on trunk too (seems to be the JVM that don't exit properly). And there's also a few serializationTest failing but it seems to be more related to the fact that the patch bumps the messaging version up that anything else. I'll look at that later.
        • our old functional tests (in test/system) pass. Again, there is a few failure, but those are test that are assuming CollatingOrderedPartitioner (apparently nobody ran those tests in a while). Anyway, those tests test the thrift API for super columns fairly thorougly.
        • you can now access super column family from CQL3.
        • I've also (briefly) tested wire compatibily and that you can do super columns queries in a mixed version cluster.

        Regarding the CQL3 support, SCF for which column_metadata has been defined on the subcolumn are handled almost like sparse CF. The almost is because I've made sure we don't write row marker as in the case of sparse CF, cause that would break backward compatibility (there is no way to have a column with an empty name in a super column). For the same reason, collection are not supported either.

        One small downside that I need to note is that during upgrade from 1.2 to 2.0, there might be a noticeable latency increase in super column queries. The reason is that any read query that mix pre and post SC nodes will have a digest mismatch (and so will re-query with the full data). Indeed, digest are not versioned and cannot really be (not easily at least).

        Show
        Sylvain Lebresne added a comment - Attached patches for this at https://github.com/pcmanus/cassandra/commits/3237-1 . This ain't small so I'll try to explain the main idea here. The main idea is that internally, super column families are handled for almost all intents and purposes as if their comparator was a simple CompositeType with 2 components: the 1st one is the old super column name, the 2nd one the old sub-column name. Meaning that they are largely not a special anymore and all the super column specific code go away (including SuperColumn.java). Now for compatibility sake, the main action is in the new SuperColumns.java class. This class contains a bunch of static methods that: deserialize old super column format directly into new composite based CF. serialize new composite based CF to the old super column format convert 'super column query filters' to and from 'composite based query filters'. Then in ColumnFamilySerializer and the ReadCommand serializer, we use those static methods when talking to old nodes (and a super column family is involved). We also convert thrift SC queries into equivalent ones on the new composite format in CassandraServer.java. The patch also don't shy away from removing abstractions that are not necessary anymore once super columns are removed. Most notably: QueryPath is removed. It was honestly already kind of useless with super columns but even more so without them. It was also error-prone imho because some method that were taking a QueryPath were actually ignoring everything except the columnFamilyName for instance. I note that the class itself is not removed but kept only to simplify wire compatibility with old nodes. IColumn and IColumnContainer are removed. We could also merge ColumnFamily and AbstractColumnContainer but I've left that to later. As far as testing goes: the unit tests pass more or less. There's CassandraServerTest that timeout on my box, but it does so on trunk too (seems to be the JVM that don't exit properly). And there's also a few serializationTest failing but it seems to be more related to the fact that the patch bumps the messaging version up that anything else. I'll look at that later. our old functional tests (in test/system) pass. Again, there is a few failure, but those are test that are assuming CollatingOrderedPartitioner (apparently nobody ran those tests in a while). Anyway, those tests test the thrift API for super columns fairly thorougly. you can now access super column family from CQL3. I've also (briefly) tested wire compatibily and that you can do super columns queries in a mixed version cluster. Regarding the CQL3 support, SCF for which column_metadata has been defined on the subcolumn are handled almost like sparse CF. The almost is because I've made sure we don't write row marker as in the case of sparse CF, cause that would break backward compatibility (there is no way to have a column with an empty name in a super column). For the same reason, collection are not supported either. One small downside that I need to note is that during upgrade from 1.2 to 2.0, there might be a noticeable latency increase in super column queries. The reason is that any read query that mix pre and post SC nodes will have a digest mismatch (and so will re-query with the full data). Indeed, digest are not versioned and cannot really be (not easily at least).
        Hide
        Jonathan Ellis added a comment -

        Vijay, can you review?

        Show
        Jonathan Ellis added a comment - Vijay, can you review?
        Hide
        Vijay added a comment -

        Will do thanks!

        Show
        Vijay added a comment - Will do thanks!
        Hide
        Vijay added a comment -

        Have been staring at the patch for a while now...
        +1, LGTM (We should fix the test though).

        Show
        Vijay added a comment - Have been staring at the patch for a while now... +1, LGTM (We should fix the test though).
        Hide
        Sylvain Lebresne added a comment -

        Alright then, committed, thanks.

        (I've fixed the serialization unit tests too)

        Show
        Sylvain Lebresne added a comment - Alright then, committed, thanks. (I've fixed the serialization unit tests too)
        Hide
        André Cruz added a comment -

        As an example, supposing I have an older schema with SuperColumns like this:

        ROWKEY:
          SC1:
            C1: val
            C2: val
          SC2:
            C3: val
            C4: val
        

        With this schema I can query cassandra for "the first object in row ROWKEY", which would return:

         SC1: (C1: val, C2: val)
        

        It seems that converting this to CompositeType columns the schema would look like:

        ROWKEY:
          SC1:C1: val
          SC1:C2: val
          SC2:C3: val
          SC2:C4: val
        

        So using this converted schema, if I wanted to obtain the first object of that row, it seems that I would obtain:

        SC1:C1: val
        

        How is this converted so that compatibility is preserved?

        Thanks.

        Show
        André Cruz added a comment - As an example, supposing I have an older schema with SuperColumns like this: ROWKEY: SC1: C1: val C2: val SC2: C3: val C4: val With this schema I can query cassandra for "the first object in row ROWKEY", which would return: SC1: (C1: val, C2: val) It seems that converting this to CompositeType columns the schema would look like: ROWKEY: SC1:C1: val SC1:C2: val SC2:C3: val SC2:C4: val So using this converted schema, if I wanted to obtain the first object of that row, it seems that I would obtain: SC1:C1: val How is this converted so that compatibility is preserved? Thanks.
        Hide
        Sylvain Lebresne added a comment -

        How is this converted so that compatibility is preserved?

        Exactly the way you've described it. But the code also convert queries on super columns, so that the query that select the first super column of a row still return all the super column, not just the first subcolumn.

        Show
        Sylvain Lebresne added a comment - How is this converted so that compatibility is preserved? Exactly the way you've described it. But the code also convert queries on super columns, so that the query that select the first super column of a row still return all the super column, not just the first subcolumn.
        Hide
        André Cruz added a comment -

        Ah, great. But sorry to insist, it's just that I'm trying to convert my schemas away from SCF, and so I'm doing manually what this patch does automatically. I would like to know how can I query this CompositeType model to obtain those SCF compatible results. Can it be done with just one query?

        Say, if I wanted the first 2 SuperColumns, so I was expecting all SC1 and SC2 data, can I query Cassandra for the first 2 distinct values of the first component of a CompositeType column?

        Thanks again.

        Show
        André Cruz added a comment - Ah, great. But sorry to insist, it's just that I'm trying to convert my schemas away from SCF, and so I'm doing manually what this patch does automatically. I would like to know how can I query this CompositeType model to obtain those SCF compatible results. Can it be done with just one query? Say, if I wanted the first 2 SuperColumns, so I was expecting all SC1 and SC2 data, can I query Cassandra for the first 2 distinct values of the first component of a CompositeType column? Thanks again.
        Hide
        Sylvain Lebresne added a comment -

        and so I'm doing manually what this patch does automatically

        I'm afraid you cannot do exactly what the patch does, not at the moment at least, because said patch uses a way to count results that CQL3 uses and that is not exposed to thrift (see CASSANDRA-4989).

        it's just that I'm trying to convert my schemas away from SCF

        I know there is a lot of "don't use super columns" floating out there, but honestly on the thrift side, the super column API is probably going to be more convenient if your use case do map well to them. And while SC do have limitations like 'they are always deserialized in their entirety', this patch fixes a good part of them moving forward. Just saying.

        Show
        Sylvain Lebresne added a comment - and so I'm doing manually what this patch does automatically I'm afraid you cannot do exactly what the patch does, not at the moment at least, because said patch uses a way to count results that CQL3 uses and that is not exposed to thrift (see CASSANDRA-4989 ). it's just that I'm trying to convert my schemas away from SCF I know there is a lot of "don't use super columns" floating out there, but honestly on the thrift side, the super column API is probably going to be more convenient if your use case do map well to them. And while SC do have limitations like 'they are always deserialized in their entirety', this patch fixes a good part of them moving forward. Just saying.

          People

          • Assignee:
            Sylvain Lebresne
            Reporter:
            Matthew F. Dennis
            Reviewer:
            Vijay
          • Votes:
            13 Vote for this issue
            Watchers:
            28 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development