Blocker Description
Relying on finalizers in RocksDB has been deprecated for a long time, and starting with rocksdb 7, finalizers are removed completely (see https://github.com/facebook/rocksdb/pull/9523).
Kafka Streams currently relies on finalizers in parts to not leak memory. This needs to be resolved before we can upgrade to RocksDB 7.
See https://github.com/apache/kafka/pull/12809 .
This is a native heap profile after running Kafka Streams without finalizers for a few hours:
Total: 13547.5 MB 12936.3 95.5% 95.5% 12936.3 95.5% rocksdb::port::cacheline_aligned_alloc 438.5 3.2% 98.7% 438.5 3.2% rocksdb::BlockFetcher::ReadBlockContents 84.0 0.6% 99.3% 84.2 0.6% rocksdb::Arena::AllocateNewBlock 45.9 0.3% 99.7% 45.9 0.3% prof_backtrace_impl 8.1 0.1% 99.7% 14.6 0.1% rocksdb::BlockBasedTable::PutDataBlockToCache 6.4 0.0% 99.8% 12941.4 95.5% Java_org_rocksdb_Statistics_newStatistics___3BJ 6.1 0.0% 99.8% 6.9 0.1% rocksdb::LRUCacheShard::Insert@2d8b20 5.1 0.0% 99.9% 6.5 0.0% rocksdb::VersionSet::ProcessManifestWrites 3.9 0.0% 99.9% 3.9 0.0% rocksdb::WritableFileWriter::WritableFileWriter 3.2 0.0% 99.9% 3.2 0.0% std::string::_Rep::_S_create
Attachments
Issue Links
- links to