[CASSANDRA-6415] Snapshot repair blocks for ever if something happens to the "I made my snapshot" response

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.2.13, 2.0.4
• Labels: repair
• Severity: Normal
• Assignee: Yuki Morishita
• Reporter: Jeremiah Jordan

Authors

Yuki Morishita

Reviewers

Jonathan Ellis

• Created: 27/Nov/13 17:47
• Updated: 12/Aug/23 20:58
• Resolved: 02/Dec/13 18:52

Description

The "snapshotLatch.await();" can be waiting for ever and block all repair operations indefinitely if something happens that another node doesn't respond.

            public void makeSnapshots(Collection<InetAddress> endpoints)
            {
                try
                {
                    snapshotLatch = new CountDownLatch(endpoints.size());
                    IAsyncCallback callback = new IAsyncCallback()
                    {
                        public boolean isLatencyForSnitch()
                        {
                            return false;
                        }

                        public void response(MessageIn msg)
                        {
                            RepairJob.this.snapshotLatch.countDown();
                        }
                    };
                    for (InetAddress endpoint : endpoints)
                        MessagingService.instance().sendRR(new SnapshotCommand(tablename, cfname, sessionName, false).createMessage(), endpoint, callback);
                    snapshotLatch.await();
                    snapshotLatch = null;
                }
                catch (InterruptedException e)
                {
                    throw new RuntimeException(e);
                }
            }

Issue Links

• relates to: CASSANDRA-18748 Transient disk failure could incur snapshot repair block forever