[HDDS-11741] Recon - Implement dynamic scaling in Recon for processing of full OM snapshot by background tasks

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Recon
• Assignee: Arafat Khan
• Reporter: Devesh Kumar Singh
• Created: 18/Nov/24 06:21
• Updated: 18/Nov/24 06:21

Description

This scaling can be done only for full snapshot case, not when Recon is receiving incremental set of OM update events in every periodic sync, because incremental events in every sync can also be in huge numbers in high TPS throughput at OM due to high TP write/update workload and all those incremental events has to be processed in sequence only (already mentioned in design doc), We cannot make it concurrent processing of those incremental events and this processing by each task will take time. We cannot speed up the processing of incremental sync events.
Scaling and speed up can only be done in case of full snapshot processing. And this may improve the speed significantly when Recon is trying to catch up with OM after long pause or it was down or during initial catch up or any other case when Recon has to fall back on full snapshot processing...