[CASSANDRA-17886] Test failure: materialized_views_test.py::TestMaterializedViews::test_drop_with_stopped_build

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Fix Version/s: 4.0.x, 5.0.x, 5.x
• Component/s: Test/dtest/python
• Bug Category: Correctness - Test Failure
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Assignee: Unassigned
• Reporter: Andres de la Peña
• Created: 12/Sep/22 15:00
• Updated: 10/Jan/24 20:15

Description

The test materialized_views_test.py::TestMaterializedViews::test_drop_with_stopped_build seems to be flaky at least on 4.0 with j11:

• https://app.circleci.com/pipelines/github/adelapena/cassandra/2079/workflows/67b3a8fd-54f8-49bf-9984-722799e5db1c/jobs/20826
• https://app.circleci.com/pipelines/github/adelapena/cassandra/2079/workflows/4ad4eae4-8438-4bd6-ba9b-0ed45f6a2550/jobs/20827

test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [[node1] 'ERROR [OptionalTasks:1] 2022-09-12 14:39:20,940 CassandraDaemon.java:581 - Exception in thread Thread[OptionalTasks:1,5,main]\njava.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.cassandra.db.view.ViewBuilderTask$StoppedException: Compaction interrupted: View build(aeda4cb1-32a8-11ed-b395-8fa6c9b92701, 664 / 1000 token range parts)@a46e4ce0-32a8-11ed-b395-8fa6c9b92701(ks, t)\n\tat org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:488)\n\tat org.apache.cassandra.db.view.ViewBuilder.stop(ViewBuilder.java:236)\n\tat org.apache.cassandra.db.view.View.stopBuild(View.java:224)\n\tat org.apache.cassandra.db.view.View.build(View.java:211)\n\tat org.apache.cassandra.db.view.ViewManager.buildAllViews(ViewManager.java:180)\n\tat org.apache.cassandra.service.CassandraDaemon.lambda$setup$2(CassandraDaemon.java:433)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.util.concurrent.ExecutionException: org.apache.cassandra.db.view.ViewBuilderTask$StoppedException: Compaction interrupted: View build(aeda4cb1-32a8-11ed-b395-8fa6c9b92701, 664 / 1000 token range parts)@a46e4ce0-32a8-11ed-b395-8fa6c9b92701(ks, t)\n\tat com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)\n\tat com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:533)\n\tat com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:90)\n\tat org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:484)\n\t... 12 common frames omitted\nCaused by: org.apache.cassandra.db.view.ViewBuilderTask$StoppedException: Compaction interrupted: View build(aeda4cb1-32a8-11ed-b395-8fa6c9b92701, 664 / 1000 token range parts)@a46e4ce0-32a8-11ed-b395-8fa6c9b92701(ks, t)\n\tat org.apache.cassandra.db.view.ViewBuilderTask.finish(ViewBuilderTask.java:194)\n\tat org.apache.cassandra.db.view.ViewBuilderTask.call(ViewBuilderTask.java:171)\n\tat org.apache.cassandra.db.compaction.CompactionManager.lambda$submitViewBuilder$6(CompactionManager.java:1802)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$FutureLocalSessionWrapper.lambda$new$0(DebuggableThreadPoolExecutor.java:408)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\t... 4 common frames omitted']

This error has been found while testing CASSANDRA-17819: https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/281/workflows/6ae07ab4-8acb-4414-b041-88d776595703/jobs/2121/tests

The repeated runs have been generated with:

.circleci/generate.sh -m \
  -e REPEATED_DTEST_COUNT=200 \
  -e REPEATED_DTEST_NAME=materialized_views_test.py::TestMaterializedViews::test_drop_with_stopped_build

Flakiness seems ~0.75%. The failure hasn't been yet seen on Butler.

Issue Links

• Discovered while testing: CASSANDRA-17819 Test failure: org.apache.cassandra.distributed.test.SchemaTest.schemaReset