[SAMZA-2018] State restore improvements

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1
• Assignee: Unassigned
• Reporter: Rayman
• Created: 01/Dec/18 01:04
• Updated: 22/Mar/19 23:38
• Resolved: 19/Mar/19 17:12

Description

• Consumer consolidation to ensure 1 storeConsumer per system, earlier it was 1 consumer per SSP per store.
• Refactoring stores to use ContainerStorageManager with parallelization for restoration, and serial execution of sysConsumers start, stop, register, etc.
• Making changes so that Samza can use some of RocksDBs benefits such as using bulk loading options.

Issue Links

• links to: GitHub Pull Request #823
• links to: GitHub Pull Request #843
• links to: GitHub Pull Request #864