Blocker Description
Updating certain dynamic configurations (for example `message.max.bytes`) causes retention based on time to reset to the default value (source code) for log.retention.ms. This poses a durability issue if users have set their retention by using log.retention.hours or log.retention.minutes. In other words, if a user has set log.retention.hours=-1 (infinite retention) and they dynamically change `message.max.bytes` their retention will immediately change back to the default of 604800000 ms (7 days) and data before this will be scheduled for deletion immediately.
Steps to reproduce:
1. Add log.retention.minutes=1,log.retention.check.interval.ms=1000 to server.properties
2. Start a single ZK or KRaft instance + a single Kafka instance
3. Create a topic using
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic A --replication-factor 1 --partitions 1 --config min.insync.replicas=1 --config segment.bytes=512
4. Create a few segments with the console producer
5. Observe that they are deleted after 1 minute
6. Use the following command
bin/kafka-configs.sh --bootstrap-server loclahost:9092 --entity-type brokers --entity-default --alter --add-config message.max.bytes=1048609
(the value of `message.max.bytes` is irrelevant)
7. Create a few more segments with the console producer
8. Observe that segments are no longer deleted after 1 minute
Attachments
Issue Links
- is duplicated by
-
KAFKA-15266 Static configs set for non primary synonyms are ignored for Log configs
-
- Resolved
-
- links to