[CASSANDRA-9097] Repeated incremental nodetool repair results in failed repairs due to running anticompaction

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.1.6, 2.2.0 beta 1
• Component/s: Legacy/Streaming and Messaging
• Severity: Low
• Assignee: Yuki Morishita
• Reporter: Gustav Munkby

Authors

Yuki Morishita

Reviewers

Marcus Eriksson

• Created: 02/Apr/15 10:51
• Updated: 16/Apr/19 09:31
• Resolved: 15/May/15 16:04

Description

I'm trying to synchronize incremental repairs over multiple nodes in a Cassandra cluster, and it does not seem to easily achievable.

In principle, the process iterates through the nodes of the cluster and performs `nodetool -h $NODE repair --incremental`, but that sometimes fails on subsequent nodes. The reason for failing seems to be that the repair returns as soon as the repair and the local anticompaction has completed, but does not guarantee that remote anticompactions are complete. If I subsequently try to issue another repair command, they fail to start (and terminate with failure after about one minute). It usually isn't a problem, as the local anticompaction typically involves as much (or more) data as the remote ones, but sometimes not.

Issue Links

• depends upon: CASSANDRA-9151 Anti-compaction is blocking ANTI_ENTROPY stage
• is part of: CASSANDRA-9143 Fix consistency of incrementally repaired data across replicas