[CASSANDRA-7705] Safer Resource Management

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.3, 2.2.0 beta 1
• Assignee: Benedict Elliott Smith
• Reporter: Benedict Elliott Smith

Authors

Benedict Elliott Smith

Reviewers

Marcus Eriksson

• Created: 06/Aug/14 13:15
• Updated: 16/Apr/19 09:31
• Resolved: 28/Jan/15 15:22

Description

We've had a spate of bugs recently with bad reference counting. these can have potentially dire consequences, generally either randomly deleting data or giving us infinite loops.

Since in 2.1 we only reference count resources that are relatively expensive and infrequently managed (or in places where this safety is probably not as necessary, e.g. SerializingCache), we could without any negative consequences (and only slight code complexity) introduce a safer resource management scheme for these more expensive/infrequent actions.

Basically, I propose when we want to acquire a resource we allocate an object that manages the reference. This can only be released once; if it is released twice, we fail immediately at the second release, reporting where the bug is (rather than letting it continue fine until the next correct release corrupts the count). The reference counter remains the same, but we obtain guarantees that the reference count itself is never badly maintained, although code using it could mistakenly release its own handle early (typically this is only an issue when cleaning up after a failure, in which case under the new scheme this would be an innocuous error)

Issue Links

• is duplicated by: CASSANDRA-8699 SSTableRewriter.abort() can throw an exception when closing the open writer
• is related to: CASSANDRA-7000 Assertion in SSTableReader during repair.
• is related to: CASSANDRA-7220 Nodes hang with 100% CPU load
• is related to: CASSANDRA-7704 FileNotFoundException during STREAM-OUT triggers 100% CPU usage
• is related to: CASSANDRA-8061 tmplink files are not removed
• is related to: CASSANDRA-7262 During streaming: java.lang.AssertionError: Reference counter -1
• relates to: CASSANDRA-8386 Make sure we release references to sstables after incremental repair
• relates to: CASSANDRA-8513 SSTableScanner may not acquire reference, but will still release it when closed
• relates to: CASSANDRA-8506 Improve management of DataTracker, esp. compacting