[CASSANDRA-9561] Lifecycle unit tests fail when run in parallel

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.2.0 rc1, 3.0 alpha 1
• Component/s: Legacy/Local Write-Read Paths, Legacy/Testing
• Assignee: Stefania Alborghetti
• Reporter: Stefania Alborghetti

Authors

Stefania Alborghetti

Reviewers

Benedict Elliott Smith

• Created: 06/Jun/15 05:34
• Updated: 16/Apr/19 09:31
• Resolved: 07/Jun/15 12:37

Description

We had a new failure in TrackerTest:

http://cassci.datastax.com/view/trunk/job/trunk_testall/139/testReport/junit/org.apache.cassandra.db.lifecycle/TrackerTest/testDropSSTables/

14:02:05     [junit] Testcase: testSSTablesInBounds(org.apache.cassandra.db.lifecycle.ViewTest):	FAILED
14:02:05     [junit] 0(true) 1(false) expected:<1> but was:<2>
14:02:05     [junit] junit.framework.AssertionFailedError: 0(true) 1(false) expected:<1> but was:<2>
14:02:05     [junit] 	at org.apache.cassandra.db.lifecycle.ViewTest.testSSTablesInBounds(ViewTest.java:65)
14:02:05     [junit] 
14:02:05     [junit] 
14:02:05     [junit] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.58 sec
14:02:05     [junit] 
14:02:05     [junit] ------------- Standard Output ---------------
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-1-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-1-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-2-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-2-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-0-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ERROR 13:57:36 Unable to delete /tmp/la-0-big-Data.db (it will be removed on server restart; we'll also retry after GC)
14:02:05     [junit] ------------- ---------------- ---------------
14:02:05     [junit] Testcase: testDropSSTables(org.apache.cassandra.db.lifecycle.TrackerTest):	FAILED
14:02:05     [junit] expected:<9> but was:<95>
14:02:05     [junit] junit.framework.AssertionFailedError: expected:<9> but was:<95>
14:02:05     [junit] 	at org.apache.cassandra.db.lifecycle.TrackerTest.testDropSSTables(TrackerTest.java:217)
14:02:05     [junit] 	at org.apache.cassandra.db.lifecycle.TrackerTest.testDropSSTables(TrackerTest.java:192)

I could reproduce it easily but only if running the tests in parallel:

ant test -Dtest.runners=4 -Dtest.name=lifecycle/*Test

If running with a single JVM then no failures can be detected, not even after 50 runs in a loop.

I think the problem is a collision of file names for the sstables created by MockSchema. When a JVM exits files are deleted due to File.deleteOnExit(), and a test running in a different JVM and using the same file names may fail. This is the only explanation I could think of.

I propose to place sstable files created by MockSchema in folders unique to each test by passing a namespace to MockSchema, equal to the test name.

cc benedict

Issue Links

• is related to: CASSANDRA-9514 TrackerTest.testAddSSTables fails in 2.2 and trunk
• links to: Trunk patch