[CASSANDRA-14762] Transient node receives full data requests in dtests

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-alpha1, 4.0
• Component/s: Feature/Transient Replication
• Severity: Normal
• Assignee: Benedict Elliott Smith
• Reporter: Ariel Weisberg

Authors

Benedict Elliott Smith

Reviewers

Ariel Weisberg

• Created: 18/Sep/18 17:42
• Updated: 15/May/20 08:01
• Resolved: 03/Oct/18 13:29

Description

I saw this running them on my laptop with rapid write protection disabled. Attached is a patch for disabling rapid write protection in the transient dtests.

.Exception in thread Thread-19:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Users/aweisberg/repos/cassandra-dtest/venv/src/ccm/ccmlib/cluster.py", line 180, in run
    self.scan_and_report()
  File "/Users/aweisberg/repos/cassandra-dtest/venv/src/ccm/ccmlib/cluster.py", line 173, in scan_and_report
    on_error_call(errordata)
  File "/Users/aweisberg/repos/cassandra-dtest/dtest_setup.py", line 137, in _log_error_handler
    pytest.fail("Error details: \n{message}".format(message=message))
  File "/Users/aweisberg/repos/cassandra-dtest/venv/lib/python3.6/site-packages/_pytest/outcomes.py", line 96, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: Error details:
Errors seen in logs for: node3
node3: ERROR [ReadStage-1] 2018-09-18 12:28:48,344 AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread Thread[ReadStage-1,5,main]
org.apache.cassandra.exceptions.InvalidRequestException: Attempted to serve transient data request from full node in org.apache.cassandra.db.ReadCommandVerbHandler@3c55e0ff
	at org.apache.cassandra.db.ReadCommandVerbHandler.validateTransientStatus(ReadCommandVerbHandler.java:104)
	at org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:53)
	at org.apache.cassandra.net.MessageDeliveryTask.process(MessageDeliveryTask.java:92)
	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:54)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:110)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

Issue Links

• is part of: CASSANDRA-14697 Transient Replication 4.0 pre-release followup work