[KAFKA-2159] offsets.topic.segment.bytes and offsets.topic.retention.minutes are ignored

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.8.2.1
• Component/s: offset manager
• Labels: newbie
• Assignee: Manikumar
• Reporter: Rafał Boniecki
• Reviewer: Joel Jacob Koshy
• Created: 30/Apr/15 13:35
• Updated: 17/May/16 12:52
• Resolved: 17/May/16 12:52

Description

My broker configuration:

offsets.topic.num.partitions=20
offsets.topic.segment.bytes=10485760
offsets.topic.retention.minutes=10080

Describe of __consumer_offsets topic:

Topic:__consumer_offsets PartitionCount:20 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact
Topic: __consumer_offsets Partition: 0 Leader: 112 Replicas: 112,212,312 Isr: 212,312,112
Topic: __consumer_offsets Partition: 1 Leader: 212 Replicas: 212,312,412 Isr: 212,312,412
Topic: __consumer_offsets Partition: 2 Leader: 312 Replicas: 312,412,512 Isr: 312,412,512
Topic: __consumer_offsets Partition: 3 Leader: 412 Replicas: 412,512,112 Isr: 412,512,112
Topic: __consumer_offsets Partition: 4 Leader: 512 Replicas: 512,112,212 Isr: 512,212,112
Topic: __consumer_offsets Partition: 5 Leader: 112 Replicas: 112,312,412 Isr: 312,412,112
Topic: __consumer_offsets Partition: 6 Leader: 212 Replicas: 212,412,512 Isr: 212,412,512
Topic: __consumer_offsets Partition: 7 Leader: 312 Replicas: 312,512,112 Isr: 312,512,112
Topic: __consumer_offsets Partition: 8 Leader: 412 Replicas: 412,112,212 Isr: 412,212,112
Topic: __consumer_offsets Partition: 9 Leader: 512 Replicas: 512,212,312 Isr: 512,212,312
Topic: __consumer_offsets Partition: 10 Leader: 112 Replicas: 112,412,512 Isr: 412,512,112
Topic: __consumer_offsets Partition: 11 Leader: 212 Replicas: 212,512,112 Isr: 212,512,112
Topic: __consumer_offsets Partition: 12 Leader: 312 Replicas: 312,112,212 Isr: 312,212,112
Topic: __consumer_offsets Partition: 13 Leader: 412 Replicas: 412,212,312 Isr: 412,212,312
Topic: __consumer_offsets Partition: 14 Leader: 512 Replicas: 512,312,412 Isr: 512,312,412
Topic: __consumer_offsets Partition: 15 Leader: 112 Replicas: 112,512,212 Isr: 512,212,112
Topic: __consumer_offsets Partition: 16 Leader: 212 Replicas: 212,112,312 Isr: 212,312,112
Topic: __consumer_offsets Partition: 17 Leader: 312 Replicas: 312,212,412 Isr: 312,212,412
Topic: __consumer_offsets Partition: 18 Leader: 412 Replicas: 412,312,512 Isr: 412,312,512
Topic: __consumer_offsets Partition: 19 Leader: 512 Replicas: 512,412,112 Isr: 512,412,112

OffsetManager logs:

2015-04-29 17:58:43:403 CEST DEBUG [kafka-scheduler-3][kafka.server.OffsetManager] Compacting offsets cache.
2015-04-29 17:58:43:403 CEST DEBUG [kafka-scheduler-3][kafka.server.OffsetManager] Found 1 stale offsets (older than 86400000 ms).
2015-04-29 17:58:43:404 CEST TRACE [kafka-scheduler-3][kafka.server.OffsetManager] Removing stale offset and metadata for [drafts,tasks,1]: OffsetAndMetadata[824,consumer_id = drafts, time = 1430322433,0]
2015-04-29 17:58:43:404 CEST TRACE [kafka-scheduler-3][kafka.server.OffsetManager] Marked 1 offsets in [__consumer_offsets,2] for deletion.
2015-04-29 17:58:43:404 CEST DEBUG [kafka-scheduler-3][kafka.server.OffsetManager] Removed 1 stale offsets in 1 milliseconds.

Parameters are ignored and default values are used instead.

Issue Links

• is duplicated by: KAFKA-3718 propagate all KafkaConfig __consumer_offsets configs to OffsetConfig instantiation