[CASSANDRA-18814] Repair hangs on Cassandra 4.0.11

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Invalid
• Component/s: Consistency/Repair
• Platform: All
• Assignee: Unassigned
• Reporter: Hugo Torralbo
• Created: 31/Aug/23 13:12
• Updated: 31/Aug/23 13:57
• Resolved: 31/Aug/23 13:57

Description

When we run a full repair on Cassandra 4.0.11, it hangs and doesn't evolve. What we noticed was this message:

"WARN [Messaging-OUT-/214.5.143.5:7001->/214.5.143.4:7001-LARGE_MESSAGES] 2023-08-18 07:02:54,862 OutboundConnection.java:488 - /214.5. 143.5:7001->/214.5.143.4:7001-LARGE_MESSAGES-[no-channel] dropping message of type VALIDATION_RSP due to error
java.nio.channels.ClosedChannelException: null"

in one of the nodes, in the validate phase the merkle tree.

Besides that, I found some connection reset , but we do not know if there is a relation it.

_18 07:02:54,860 OutboundConnection.java:1056 - /214.5.143.5:7001->/214.5.143.4:7001-LARGE_MESSAGES-94900b4d channel closed by provider
io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer_

 

I have uploaded logs from all nodes