[CASSANDRA-10737] TestTopology.crash_during_decommission_test dtest sometimes fails due to interrupted streaming

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Auto Closed
• Component/s: Test/dtest/python
• Severity: Low
• Assignee: Unassigned
• Reporter: Joel Knighton
• Created: 19/Nov/15 18:08
• Updated: 13/Sep/21 17:12
• Resolved: 13/Sep/21 17:12

Description

Example here.

The test fails due to an unexpected error in the log as follows:

Unexpected error in node1 node log: ['ERROR [STREAM-IN-/127.0.0.2] 2015-10-29 10:33:42,030 StreamSession.java:520 - [Stream #862cc220-7e28-11e5-b94f-5f64a052c605] Streaming error occurred java.io.IOException: Connection reset by peer \tat sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_45] \tat sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_45] \tat sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_45] \tat sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.8.0_45] \tat sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_45] \tat org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54) ~[main/:na] \tat org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:261) ~[main/:na] \tat java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]', 'ERROR [STREAM-OUT-/127.0.0.2] 2015-10-29 10:33:42,031 StreamSession.java:520 - [Stream #862cc220-7e28-11e5-b94f-5f64a052c605] Streaming error occurred']

This is because a node is crashed while it is receiving a stream from the decommissioned node.

We should ignore this error and possibly handle this hung leaving state by restarting the decommission.

Issue Links

• relates to: CASSANDRA-10231 Null status entries on nodes that crash during decommission of a different node