[CASSANDRA-19549] Test failure: rebuild_test.TestRebuild.test_resumable_rebuild

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-rc1, 5.0, 5.1
• Component/s: Test/dtest/python
• Bug Category: Correctness - Test Failure
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: NA

Source Control Link

https://github.com/apache/cassandra-dtest/commit/131982f44a48e3d24e6ebba4a4884fd5d5948218

Test and Documentation Plan

run CI

• Assignee: Brandon Williams
• Reporter: Jacek Lewandowski

Authors

Brandon Williams

Reviewers

Stefan Miklosovic

• Created: 10/Apr/24 14:35
• Updated: 20/Jun/24 18:52
• Resolved: 20/Jun/24 18:52

Description

Interrupted exception thrown during shutdown and caught by JVMStabilityInspector - does not look serious but we may want to ignore interrupted exception during shutdown.

https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/1326/workflows/021d350a-4b62-44af-9650-f5a0eb105522/jobs/70413/tests

failed on teardown with "Unexpected error found in node logs (see stdout for full details). 
Errors: [[node2] 'ERROR [NettyStreaming-Outbound-/127.0.0.3.7000:3] 2024-04-09 08:32:19,662 JVMStabilityInspector.java:70 - Exception in thread Thread NettyStreaming-Outbound-/127.0.0.3.7000:3,5,NettyStreaming-Outbound-/127.0.0.3.7000]
org.apache.cassandra.utils.concurrent.UncheckedInterruptedException: java.lang.InterruptedException
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:373)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.run(StreamingMultiplexedChannel.java:309)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException: null
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1081)
	at java.base/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:592)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:356)
	... 8 common frames omitted', [node2] 'ERROR [NettyStreaming-Outbound-/127.0.0.3.7000:3] 2024-04-09 08:32:19,664 ExecutionFailure.java:72 - Unexpected error while handling unexpected error
org.apache.cassandra.utils.concurrent.UncheckedInterruptedException: java.lang.InterruptedException
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:142)
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:170)
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:89)
	at org.apache.cassandra.utils.JVMStabilityInspector.uncaughtException(JVMStabilityInspector.java:78)
	at org.apache.cassandra.concurrent.ExecutionFailure.handle(ExecutionFailure.java:67)
	at org.apache.cassandra.concurrent.FutureTask.tryFailure(FutureTask.java:86)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:75)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException: null
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1081)
	at java.base/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:592)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:356)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.run(StreamingMultiplexedChannel.java:309)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
	... 4 common frames omitted']"
Unexpected error found in node logs (see stdout for full details). Errors: [[node2] 'ERROR [NettyStreaming-Outbound-/127.0.0.3.7000:3] 2024-04-09 08:32:19,662 JVMStabilityInspector.java:70 - Exception in thread Thread[NettyStreaming-Outbound-/127.0.0.3.7000:3,5,NettyStreaming-Outbound-/127.0.0.3.7000]
org.apache.cassandra.utils.concurrent.UncheckedInterruptedException: java.lang.InterruptedException
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:373)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.run(StreamingMultiplexedChannel.java:309)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException: null
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1081)
	at java.base/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:592)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:356)
	... 8 common frames omitted', [node2] 'ERROR [NettyStreaming-Outbound-/127.0.0.3.7000:3] 2024-04-09 08:32:19,664 ExecutionFailure.java:72 - Unexpected error while handling unexpected error
org.apache.cassandra.utils.concurrent.UncheckedInterruptedException: java.lang.InterruptedException
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:142)
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:170)
	at org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:89)
	at org.apache.cassandra.utils.JVMStabilityInspector.uncaughtException(JVMStabilityInspector.java:78)
	at org.apache.cassandra.concurrent.ExecutionFailure.handle(ExecutionFailure.java:67)
	at org.apache.cassandra.concurrent.FutureTask.tryFailure(FutureTask.java:86)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:75)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException: null
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1081)
	at java.base/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:592)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.acquirePermit(StreamingMultiplexedChannel.java:356)
	at org.apache.cassandra.streaming.async.StreamingMultiplexedChannel$FileStreamTask.run(StreamingMultiplexedChannel.java:309)
	at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
	at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
	... 4 common frames omitted']

Issue Links

• is related to: CASSANDRA-18856 Test failure: rebuild_test.TestRebuild.test_resumable_rebuild