[IGNITE-7182] Slow sorting of pages collection on checkpoint begin can cause zero dropdown even with throttling enabled

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 2.3
• Component/s: persistence
• Assignee: Dmitry Pavlov
• Reporter: Ivan Rakov
• Created: 13/Dec/17 11:45
• Updated: 17/Sep/19 17:45
• Resolved: 17/Sep/19 17:45

Description

Tests show that GridCacheDatabaseSharedManager#splitAndSortCpPagesIfNeeded call can last several seconds on nodes with big amount of memory (>10GB). We should optimize sorting algorithm, possibly making it multithreaded.
Another option to make pages write throttling more smooth is to get rid of this heuristic:

                // Starting with 0.05 to avoid throttle right after checkpoint start
                // 7/12 is maximum ratio of dirty pages
                dirtyRatioThreshold = (dirtyRatioThreshold * 0.95 + 0.05) * 7 / 12;

We should replace "magic" lower bound 0.05 * 7 / 12 with the real percentage of dirty pages at the moment of GridCacheDatabaseSharedManager.Checkpointer#markCheckpointBegin call return.

Issue Links

• relates to: IGNITE-7175 Throttling is not applied to page allocation
• relates to: IGNITE-7507 Ignite node performance drop during checkpoint start: store metapage eviction causes long checkpoint lock hold time
• relates to: IGNITE-6334 Throttle writing threads during ongoing checkpoint with token bucket algorithm
• links to: GitHub Pull Request #3830
• links to: GitHub Pull Request #3830