By design, the cassandra-cli caches the schema and doesn't refresh it when various commands like "describe keyspaces" are run. This is reasonable, and it is easy enough to restart the cli if necessary. However, this does lead to confusion since a new user can reasonably assume that describe keyspaces will always show an accurate current represention of the ring. We should find a way to reduce the surprise (and lack of easy discoverability) of this behaviour.
I propose any one of the following(#1 is probably the easiest and most likely):
1) Add a command (that would be documented in the cli's help) to explicitly refresh the schema ("schema refresh", "refresh schema", or anything similar).
2) Always force a refresh of the schema when performing at least the "describe keyspaces" command.
3) Add a flag to cassandra-cli to explicitly enable schema caching. If that flag is not passed, then schema caching will be disabled for that session. This suggestion assumes that for simple deployments (few CFs, etc), schema caching isn't very important to the performance of the cli.