[IGNITE-16878] Use exponential moving average to estimate checkpoint write speed

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: persistence
• Ignite Flags: Release Notes Required
• Assignee: Unassigned
• Reporter: Roman Puchkovskiy
• Created: 21/Apr/22 07:28
• Updated: 21/Apr/22 07:28

Description

The speed-based throttling (implemented in  SpeedBasedMemoryConsumptionThrottlingStrategy) uses an estimation of checkpoint write speed (in pages per second) to decide whether to throttle (and how hard) or not.

Currently, this estimation is obtained by taking a moving average for the current checkpoint and up to 3 preceeding checkpoints. This should work well if the checkpoint writing speed is stable. But if this speed changes a lot (probably, due to some other disk activity, like rotating the WAL or activity of other applications), this change only becomes noticed after a substantial amount of time.

The idea is to use exponential moving average https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average to make the metric note the changes faster. Other weightings can also be considered.