[CASSANDRA-14530] Unable to start repair process

• Type: Bug
• Status: Resolved
• Priority: Urgent
• Resolution: Fixed
• Component/s: Consistency/Repair, Legacy/Core, Local/Compaction

Environment

Description: Debian GNU/Linux 8.5 (jessie)
Codename: jessie

16GB mem. 6 allocated by java only.

Linux XXXX04 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02) x86_64 GNU/Linux

cassandra 3.11.2

 

Output of command

XProf file attached below

XXXXXXXXX04:/var/lib/cassandra# nodetool repair archive statuses -full
[2018-06-19 17:17:38,170] Starting repair command #1 (84fd0590-73cb-11e8-9815-6d9cda004dd5), repairing keyspace archive with repair options (parallelism: parallel, primary range: false, incremental: false, job threads: 1, ColumnFamilies: [statuses], dataCenters: [], hosts: [], # of ranges: 967, pull repair: false)
Exception occurred during clean-up. java.lang.reflect.UndeclaredThrowableException
Cassandra has shutdown.
error: [2018-06-19 17:17:42,233] JMX connection closed. You should check server log for repair status of keyspace archive(Subsequent keyspaces are not going to be repaired).
-- StackTrace --
java.io.IOException: [2018-06-19 17:17:42,233] JMX connection closed. You should check server log for repair status of keyspace archive(Subsequent keyspaces are not going to be repaired).
at org.apache.cassandra.tools.RepairRunner.handleConnectionFailed(RepairRunner.java:98)
at org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:86)
at javax.management.NotificationBroadcasterSupport.handleNotification(NotificationBroadcasterSupport.java:275)
at javax.management.NotificationBroadcasterSupport$SendNotifJob.run(NotificationBroadcasterSupport.java:352)
at javax.management.NotificationBroadcasterSupport$1.execute(NotificationBroadcasterSupport.java:337)
at javax.management.NotificationBroadcasterSupport.sendNotification(NotificationBroadcasterSupport.java:248)
at javax.management.remote.rmi.RMIConnector.sendNotification(RMIConnector.java:441)
at javax.management.remote.rmi.RMIConnector.access$1200(RMIConnector.java:121)
at javax.management.remote.rmi.RMIConnector$RMIClientCommunicatorAdmin.gotIOException(RMIConnector.java:1531)
at javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1352)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchOneNotif(ClientNotifForwarder.java:655)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:607)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:471)
at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:452)
at com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:108)

• Severity: Critical
• Assignee: Unassigned
• Reporter: ALEXEY KASHINTSEV
• Created: 19/Jun/18 14:24
• Updated: 16/Apr/19 09:29
• Resolved: 21/Jun/18 11:55

Description

 

Then repair starts, cassandra goes down.