[IGNITE-18132] Introduce scale down scheduler

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: Distribution Zones [Part1]
• Assignee: Mirza Aliev
• Reporter: Alexander Lapin
• Reviewer: Alexander Lapin
• Created: 11/Nov/22 13:29
• Updated: 04/Sep/24 16:19
• Resolved: 08/Feb/23 16:56

Description

Motivation

Surprisingly it's a follow up of an Introduce scale up scheduler

Following logic should be implemented 

-1	start Node A; start Node B; start Node C; CREATE ZONE zone1 WITH   DATA_NODES_AUTO_ADJUST_SCALE_UP = 300, DATA_NODES_AUTO_ADJUST_SCALE_DOWN= 300_000; CREATE TABLE Accounts … WITH  PRIMARY_ZONE = zone1	User starts three Ignite nodes A, B, C and creates table Accounts specifying scale up auto adjust timeout as 300 seconds. Accounts table is created on current topology, meaning that <Transaction>.dataNodes = [A,B,C]
0	start Node D ->     Node D join/validation ->     D enters logical topology ->     logicalTopology.onNodeAdded(Node D) ->     scale_up_auto_adjust(300) timer is     scheduled for the <Accounts> table.	At time 0 seconds the user starts one more Ignite node D, that joins the cluster. On entering logical topology the onNodeAdded event is fired. This event, schedules a timer of 300 seconds for table Accounts after which the dataNodes of that table will be recalculated from [A,B,C] to [A,B,C,D]
100	stop Node C ->      scale_down_auto_adjust(300_000) timer     is scheduled for the <Accounts> table.	At 100 seconds the user stops Node C (or it painfully dies). TableManager detects onNodeLeft(Node C) event and starts scale_down time for 300_000 seconds for table <Accounts>. Please pay attention that the node left doesn’t affect the scale_up timer.
250	start Node E ->     scale_up_auto_adjust(300) timer is     re-scheduled for the <Accounts> table.	At 250 seconds Node E is added, that re-schedules scale_up_auto_adjust timer for another 300 seconds. The important part here is that adding the node doesn’t change scale_down time only scale_up one.
550	scale_up_auto_adjust fired ->     set table.<Accounts>.dataNodes =      [NodeA, NodeB, {*}NodeC{*}, Node D, Node E]	At 550 seconds scale_up_time is fired, that leads to <Transaction>dataNodes recalculation by attaching the nodes that were added to logical topology - Nodes D and E in the given example. Please pay attention that despite the fact there's no Node C in logical topology it still takes its place in <Transaction>.dataNodes.
300100	scale_down_auto_adjust fired ->      set table.<Accounts>.dataNodes =      [NodeA, NodeB, Node D, Node E]	At 300_100 seconds scale_down_auto_adjust timer is fired, that leads to removing Node C from <Transaction>.dataNodes.

Definition of Done

• DataNodes recalculation transitively triggered by node left event is delayed for dataNodesAutoAdjustScaleDown value.
• In case of new scale down toplogy event, existing scale down timers should be re-scheduled.
• Scale up timers should not be affected.

Implementation Notes

As as for scale up.

Issue Links

• blocks: IGNITE-18134 Introduce common auto adjust scheduler
• is blocked by: IGNITE-18121 Introduce scale up scheduler
• links to: GitHub Pull Request #1631