[KAFKA-5174] RocksDb might stall in environments with 1 core only

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.2.1
• Fix Version/s: 0.11.0.0
• Component/s: streams
• Assignee: Eno Thereska
• Reporter: Eno Thereska
• Created: 04/May/17 19:19
• Updated: 08/May/17 16:42
• Resolved: 08/May/17 16:41

Description

• there is a regression in 0.10.2.1 that impacts RocksDb's behaviour (details below for first impact) from this PR: https://github.com/apache/kafka/commit/f9660e16dafb9e1109704bdecb978b7046f0cc6b#diff-046ca566243518c88e007b7499ec9f51

• worst-case impact: streams might not run out-of-box in deployments with 1 core only
• workaround: no new jars needed for the workaround, only a config change that changes a RocksDb parameter, e.g. something like in this PR: https://github.com/confluentinc/examples/pull/117/files

• full fix: add 1 to the current config (i.e., max cpus + 1) in the default settings

The RocksDb code might have a bug itself, since when running with one core
it both sets the number of compaction threads to 1 (https://github.com/facebook/rocksdb/blob/5.0.fb/util/options.cc#L762) AND the maximum number of compactions variable to 0 (https://github.com/facebook/rocksdb/blob/5.0.fb/util/options.cc#L760) which doesn't make much sense. I've sent a message in RocksDb channel.

Issue Links

• links to: GitHub Pull Request #2982