[KAFKA-3735] RocksDB objects needs to be disposed after usage

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1.0
• Component/s: streams
• Labels: architecture
• Assignee: Guozhang Wang
• Reporter: Guozhang Wang
• Created: 20/May/16 02:27
• Updated: 20/May/16 21:19
• Resolved: 20/May/16 18:52

Description

The RocksDB JNI interface RocksObject has a dispose() function which need to be explicitly triggered after it is not reused, otherwise GC may not be able to de-reference it off-heap and hence effective lead to memory leak. See: https://github.com/facebook/rocksdb/issues/752#issuecomment-146511412

We need to make sure all library-controlled RocksDB objects are disposed after usage, and also indicate users to close those out-of-control objects.

Note that RocksDB community is also going to replace the dispose API by extending AutoClosable in the future, so this ticket may need to be re-visited when upgrade RocksDB versions:

https://www.facebook.com/groups/rocksdb.dev/permalink/870848569680325/

Issue Links

• links to: GitHub Pull Request #1411