[SAMZA-675] Check backwards incompatible config changes

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.10.0
• Component/s: container
• Assignee: Unassigned
• Reporter: Naveen Somasundaram
• Created: 12/May/15 21:10
• Updated: 12/May/15 21:11

Description

One of the nice things we get for coordinator stream is, we have the previous configs we launched the job with. We can use this to our advantage for config validation. For e.g.,
1. The number of partitions in the Kafka stream has changed, this would break our change log partition mapping.
2. Store was opened with TTL, but reopened again without TTL in RocksDB, this can cause a corruption.

And many more checks like this. We need a place (probably JobCoordinator) to do these validations on, validate(oldConfig from CoordinatorStream - newConfigsUsers have passed)