> My idea was to lock a table during a definition mutation in order to prevent a row mutation from happening in progress. I still think we need this. And I think a per-table lock gives us more availability than a global table open lock.
You're using a read/write lock already pre-Multilock (iirc), so the only time Multilock matters is allowing writes to continue in KS X while modifying KS Y. Modification is rare enough, and over fast enough, that this isn't worth adding complexity for, especially when it's complexity w/ a performance penalty.
(Again, it's worth pointing out that using double-checked locking in Table.open completely bypasses the lock for existing Table objects.)
Having talked about locking strategies I will now go back to arguing that this is a solution to the wrong question. (If you are on that page already I apologize for continuing to beat that horse.
RM.apply is very very performance critical. I don't want to add extra locking there, especially not read/write lock which is more expensive.
As long as we are corruption-safe in the pedantic concurrency sense we shouldn't worry about mutations during schema modification. (Hence, I suspect that we do need to single-thread these to avoid potential bugs w/ renames.) The only op that is potentially problematic is a CF or KS rename, where ops that come in during the process can apply to the "wrong" one. But the same thing will happen to ops against the new name that get sent a ms or two earlier no matter what you do locking-wise. In other words, the client needs to quiesce that target himself anyway, so us locking during the actual fraction of a ms for the change doesn't really buy him anything.