Cassandra
  1. Cassandra
  2. CASSANDRA-1585

Support renaming columnfamilies and keyspaces

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Later
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      Renames were briefly supported but were race-prone.

        Issue Links

          Activity

          Hide
          Gary Dusbabek added a comment -

          One approach is to push migrations onto the CompactionManager executor. pro: simple. con: a major compaction on column family X will block what would normally be a quick schema update on column family Y.

          Another approach is to try to allow parallel operations, probably using locks. I envision something like this:
          lock(cfId);
          // do work
          unlock(cfId);
          pro: parallel operations. con: more complicated.

          Show
          Gary Dusbabek added a comment - One approach is to push migrations onto the CompactionManager executor. pro: simple. con: a major compaction on column family X will block what would normally be a quick schema update on column family Y. Another approach is to try to allow parallel operations, probably using locks. I envision something like this: lock(cfId); // do work unlock(cfId); pro: parallel operations. con: more complicated.
          Hide
          Gary Dusbabek added a comment -

          There are more timing issues than we discussed on IRC that neither approach addresses. They all relate to the lifecycle of ColumnFamilyStore instances. Consider these steps:

          1. Migration X is scheduled (say it's a rename)
          2. Compaction of X is scheduled.

          The compaction manager deals directly with ColumnFamilyStore instances, which aren't directly acted on by a schema migration, but which can really have their underpinnings mangled (files moving around basically). At the very least, we should check to make sure the CFS instances are still valid when the executor gets around to letting them run.

          Am I making sense?

          Show
          Gary Dusbabek added a comment - There are more timing issues than we discussed on IRC that neither approach addresses. They all relate to the lifecycle of ColumnFamilyStore instances. Consider these steps: 1. Migration X is scheduled (say it's a rename) 2. Compaction of X is scheduled. The compaction manager deals directly with ColumnFamilyStore instances, which aren't directly acted on by a schema migration, but which can really have their underpinnings mangled (files moving around basically). At the very least, we should check to make sure the CFS instances are still valid when the executor gets around to letting them run. Am I making sense?
          Hide
          Jonathan Ellis added a comment -

          Maybe an easier solution is to have rename mutate the CFS instead of unlinking it and creating a new one?

          Show
          Jonathan Ellis added a comment - Maybe an easier solution is to have rename mutate the CFS instead of unlinking it and creating a new one?
          Hide
          Jonathan Ellis added a comment -

          What to do about drop?

          Show
          Jonathan Ellis added a comment - What to do about drop?
          Hide
          Gary Dusbabek added a comment -

          Deletion is a problem too. The main problem is that there are 3 services that all aim to modify files: compaction, flushing and schema mutations. Compaction and flushing are independent, but schema mutation isn't (we don't want to flush while files are being moved or deleted).

          Show
          Gary Dusbabek added a comment - Deletion is a problem too. The main problem is that there are 3 services that all aim to modify files: compaction, flushing and schema mutations. Compaction and flushing are independent, but schema mutation isn't (we don't want to flush while files are being moved or deleted).
          Hide
          Jonathan Ellis added a comment -

          Let's do this the simple way so we can roll rc1, then we can make it better for 0.7.1: move migrations onto CompactionManager, and use Table.flusherLock.writelock to avoid problems there.

          Show
          Jonathan Ellis added a comment - Let's do this the simple way so we can roll rc1, then we can make it better for 0.7.1: move migrations onto CompactionManager, and use Table.flusherLock.writelock to avoid problems there.
          Hide
          Jonathan Ellis added a comment -

          For 0.7.0 we moved drop to the compaction manager (CASSANDRA-1631) and removed the rename operations (CASSANDRA-1630)

          Show
          Jonathan Ellis added a comment - For 0.7.0 we moved drop to the compaction manager ( CASSANDRA-1631 ) and removed the rename operations ( CASSANDRA-1630 )
          Hide
          Jonathan Ellis added a comment - - edited

          what if we named the files by cfid instead of cfname, to handle rename? we already have a name->id mapping in the system tables somewhere. (and we can assume nothing in "system" keyspace gets renamed, of course. otherwise i think we might have a chicken/egg problem.)

          Show
          Jonathan Ellis added a comment - - edited what if we named the files by cfid instead of cfname, to handle rename? we already have a name->id mapping in the system tables somewhere. (and we can assume nothing in "system" keyspace gets renamed, of course. otherwise i think we might have a chicken/egg problem.)
          Hide
          Gary Dusbabek added a comment -

          what if we named the files by cfid instead of cfname, to handle rename?

          The last time I looked, there was one part of code that still sent the CF name as part of its serialized form. I thought it was CFSerializer, but it appears to be using ids.

          I'll take a closer look to see if this is possible. If we're not playing fast and loose with CFnames in messages, I think it will work.

          Show
          Gary Dusbabek added a comment - what if we named the files by cfid instead of cfname, to handle rename? The last time I looked, there was one part of code that still sent the CF name as part of its serialized form. I thought it was CFSerializer, but it appears to be using ids. I'll take a closer look to see if this is possible. If we're not playing fast and loose with CFnames in messages, I think it will work.
          Hide
          Robert Coli added a comment -

          To anyone who is wondering about the manual way to do this :

          1) create schema for NEW_Keyspace
          2) stop writes to OLD_Keyspace from app (reads can continue)
          3) flush OLD_Keyspace on every node, via nodetool
          4) hard link all sstables from OLD_Keyspace directory to NEW_Keyspace directory
          5) call nodetool -h localhost refresh NEW_Keyspace
          6) enable reads/writes from/to NEW_Keyspace from app (disable reads on OLD_Keyspace)
          7) clean up OLD_Keyspace (drop schema, delete files, etc.)

          Alternately, if you don't want to do 2/6 because you can't tolerate OLD_Keyspace not being writable, you can enable writes to NEW_Keyspace, flush OLD_Keyspace, hard link the just-flushed tables and then enable reads from NEW_Keyspace. This resolves the delta with a shorter window where you can't write.

          The same technique could also be applied to renaming Columnfamilies, although in the Columnfamily case the files also need to be renamed. In Cassandra 1.1+, the files get renamed to include the Keyspace name, so that would have to change as appropriate.

          Show
          Robert Coli added a comment - To anyone who is wondering about the manual way to do this : 1) create schema for NEW_Keyspace 2) stop writes to OLD_Keyspace from app (reads can continue) 3) flush OLD_Keyspace on every node, via nodetool 4) hard link all sstables from OLD_Keyspace directory to NEW_Keyspace directory 5) call nodetool -h localhost refresh NEW_Keyspace 6) enable reads/writes from/to NEW_Keyspace from app (disable reads on OLD_Keyspace) 7) clean up OLD_Keyspace (drop schema, delete files, etc.) Alternately, if you don't want to do 2/6 because you can't tolerate OLD_Keyspace not being writable, you can enable writes to NEW_Keyspace, flush OLD_Keyspace, hard link the just-flushed tables and then enable reads from NEW_Keyspace. This resolves the delta with a shorter window where you can't write. The same technique could also be applied to renaming Columnfamilies, although in the Columnfamily case the files also need to be renamed. In Cassandra 1.1+, the files get renamed to include the Keyspace name, so that would have to change as appropriate.
          Hide
          Francisco Trujillo added a comment -

          For the people who is going to try the manual method describe by Robert Coli i found some difficulties (my errors but could be that someone else have the same problem):

          In 1) You have to create the scheme with all the column familys and indexes.

          In 4) remember that the files that stored the sstables start with the name of the keyspace. You have to rename the files in order to be recognized by the nodetool refresh.

          Show
          Francisco Trujillo added a comment - For the people who is going to try the manual method describe by Robert Coli i found some difficulties (my errors but could be that someone else have the same problem): In 1) You have to create the scheme with all the column familys and indexes. In 4) remember that the files that stored the sstables start with the name of the keyspace. You have to rename the files in order to be recognized by the nodetool refresh.
          Hide
          Alain RODRIGUEZ added a comment -

          Does any one knows if this feature will be available again ? It was removed almost 3 years ago and this issue is marked as "resolution: later".

          It is not a critical feature, but since CQL3 purpose is to be similar to the SQL norm, it would be interesting to be able to rename databases (keyspace) and tables (CF).

          This is feature people would use for sure.

          Show
          Alain RODRIGUEZ added a comment - Does any one knows if this feature will be available again ? It was removed almost 3 years ago and this issue is marked as "resolution: later". It is not a critical feature, but since CQL3 purpose is to be similar to the SQL norm, it would be interesting to be able to rename databases (keyspace) and tables (CF). This is feature people would use for sure.

            People

            • Assignee:
              Unassigned
              Reporter:
              Stu Hood
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development