[KAFKA-15266] Static configs set for non primary synonyms are ignored for Log configs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.6.0
• Component/s: core
• Assignee: Aman Harish Gandhi
• Reporter: Aman Harish Gandhi
• Created: 28/Jul/23 09:51
• Updated: 26/Sep/24 11:48
• Resolved: 26/Sep/24 11:48

Description

In our server.properties we had the following config

log.retention.hours=48

We noticed that after running alter configs to update broker level config(for a config unrelated to retention) we were only deleting data after 7 days instead of the configured 2.

The alterconfig we had ran was similar to this

sh kafka-config.sh --bootstrap-server localhost:9092 --alter --add-config "log.segment.bytes=500000"

Digging deeper the issue could be pin pointed to the reconfigure block of DynamicLogConfig inside DynamicBrokerConfig. Here we only look at the "primary" KafkaConfig synonym of the LogConfig and if it is not set then we remove the value set in default log config as well. This eventually leads to the retention.ms not being set in the default log config and that leads to the default value of 7 days being used. The value set in "log.retention.hours" is completely ignored in this case.

Pasting the relevant code block here

newConfig.valuesFromThisConfig.forEach { (k, v) =>
  if (DynamicLogConfig.ReconfigurableConfigs.contains(k)) {
    DynamicLogConfig.KafkaConfigToLogConfigName.get(k).foreach { configName =>
      if (v == null)
         newBrokerDefaults.remove(configName)
      else
        newBrokerDefaults.put(configName, v.asInstanceOf[AnyRef])
    }
  }
} 

In the above block `DynamicLogConfig.ReconfigurableConfigs` contains only log.retention.ms. It does not contain the other synonyms like `log.retention.minutes` or `log.retention.hours`.

This issue seems prevalent in all cases where there are more than 1 KafkaConfig synonyms for the LogConfig.

Issue Links

• duplicates: KAFKA-17584 Fix incorrect synonym handling for dynamic log configurations
• links to: GitHub Pull Request #14119