[IGNITE-19819] Lease batches compaction

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: ignite-3
• Assignee: Unassigned
• Reporter: Denis Chudov
• Created: 23/Jun/23 09:08
• Updated: 29/Mar/24 12:06

Description

Motivation 

After IGNITE-19578 leases should be stored as a single batch in meta storage. However, the size of such a batch is significant and can be reduced.

Each lease contains group name, leaseholder name, left and right timestamp and couple of boolean flags.
Many leases share the same leaseholder. Also, many leases share the same right border, as batch of leases are renewed on every iteration of lease updater and get the same right border.
So, the compacted data structure for all leases could be a map

right border -> set of leaseholders -> set of leases which contain only group name, left border and flags.

It is important that this data structure is applicable to meta storage representation, in-memory representation of leases should remain the same.

Definition of done
Amount of space required for storing leases is significantly reduced.

Implementation notes

The key should be prefix + right border. On each iteration the corresponding right border should be removed and new one put, so on each iteration there will be done just one meta storage invoke. 

To avoid ABA problem during leases' updates via invokes, entries should be versioned, this can be done by assigning unique version to each right border key. There are cases when all leaseholders can be removed from some entry and then another leaseholders added again (e.g. accepting leases and removing them from right border that matches long-term unaccepted leases, and later adding the regular leases to the same right border). In this cases the entry should not be removed from meta storage, in spite it doesn't have leases, to preserve the version of the entry.

To avoid merging of batches, lease prolongation should be changed: new right border should be calculated as current_right_border + lease_interval (now: current_time + lease_interval ).

Issue Links

• blocks: IGNITE-21142 "SqlException: Table with name 'N' already exists" upon creating a non-existing table