Major Description
We have a number of per-topic configurations that control message retention and flush interval. Here is the list of properties I find in KafkaConfig that appear to be per-topic:
topic.log.file.size
topic.log.roll.hours
topic.log.retention.hours
topic.log.retention.size
topic.flush.intervals.ms
Currently we specify these in server.properties. This is not a good solution as it requires a rolling bounce of the cluster to make a change, which just doesn't scale to having hundreds of topics. Also the map encoded in a CSV string is kind of hacky.
We should move these into ZK in some kind of JSON blob that allows easily adding new per-topic configs and we should remove these from server.properties.
It would be good to start with a wiki design and get consensus on that first.