[IGNITE-16582] Improve behavior of speed-based throttling when dirty pages ratio is low

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.12
• Fix Version/s: 2.14
• Component/s: persistence
• Release Note: Improved behavior of the speed-based throttling when a dirty-page ratio is low.
• Ignite Flags: Release Notes Required
• Assignee: Roman Puchkovskiy
• Reporter: Roman Puchkovskiy
• Reviewer: Semyon Danilov
• Created: 18/Feb/22 14:39
• Updated: 09/Sep/22 16:36
• Resolved: 04/Jun/22 08:45

Description

There is a log:

Throttling is applied to page modifications [
percentOfPartTime=0.59, 
markDirty=7424 pages/sec, 
checkpointWrite=6268 pages/sec, 
estIdealMarkDirty=0 pages/sec, 
curDirty=0.00, 
maxDirty=0.24, 
avgParkTime=79770 ns, 
pages: (total=67085, evicted=0, written=40916, synced=0, cpBufUsed=3, cpBufTotal=518215)]

Here, it can be seen that, although there are plenty of non-dirty pages, throttling is applied. This happens because our speed-based throttling has 2 algorithms for protecting non-dirty pages from exhaustion:

1. A more complex one that computes max allowable dirty ratio and ideal marking speed and throttles when both dirty ratio and current marking speed surpass these values
2. A simpler one that throttles if the current marking speed is higher than the average checkpointing speed

In the shown example the first algorithm does not throttle, but the second one does.

It looks like the throttling is enabled too early.

One way to solve this problem is to just disable the second algorithm as the first seems to be more adequate (but this needs careful consideration of all possible cases).

Another way is to consider averaged marking speed instead of (or in addition to) the current marking speed when deciding whether to throttle or not.

Issue Links

• is related to: IGNITE-14197 Checkpoint thread can't take checkpoint write lock because it waits for parked threads to complete their work
• links to: GitHub Pull Request #9924