[CASSANDRA-9434] If a node loses schema_columns SSTables it could delete all secondary indexes from the schema

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Won't Fix
• Component/s: Feature/2i Index
• Severity: Normal
• Since Version: 2.0.11
• Assignee: Unassigned
• Reporter: Richard Low
• Created: 20/May/15 01:05
• Updated: 16/Apr/19 09:31
• Resolved: 22/Jul/16 17:42

Description

It is possible that a single bad node can delete all secondary indexes if it restarts and cannot read its schema_columns SSTables. Here's a reproduction:

• Create a 2 node cluster (we saw it on 2.0.11)
• Create the schema:

create keyspace myks with replication = {'class':'SimpleStrategy', 'replication_factor':1};
use myks;
create table mytable (a text, b text, c text, PRIMARY KEY (a, b) );
create index myindex on mytable(b);

NB index must be on clustering column to repro

• Kill one node
• Wipe its commitlog and system/schema_columns sstables.
• Start it again
• Run on this node

select index_name from system.schema_columns where keyspace_name = 'myks' and columnfamily_name = 'mytable' and column_name = 'b';

and you'll see the index is null.

• Run 'describe schema' on the other node. Sometimes it will not show the index, but you might need to bounce for it to disappear.

I think the culprit is SystemKeyspace.copyAllAliasesToColumnsProper.