[IGNITE-18171] Descibe nodes start/stop scenarios - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Task
Status: Resolved
Priority: Major
Resolution: Done
Affects Version/s: None
Fix Version/s: None
Component/s: sql
Labels:
- ignite-3

Epic Link:
Grid start-stop scenarios
Ignite Flags:

Docs Required, Release Notes Required

Description

Definitions.

We can distinguish next cluster node groups, see below. Each node may be part of one or more groups.

Cluster Management Group (CMG), that control new nodes join process.
MetaStorage group (MSG), that hosts meta storage.
Data node group (DNG), that just hosts tables partitions.

The components (CMG, meta storage, tables components) are depends on each other, but may resides on different (even disjoint) node subsets. So, some components may become temporary unavailable, and dependant components must be aware of such issues and handle them (wait, retry, throw exception or whatever) in expected way, which has to be documented also.
See IEP for details

Motivation.

As of now, the correct way to start the grid (after it was stopped) is: start CMG nodes, then Meta Storage nodes, then Data nodes. And in backward order for correct stop. Other scenarios are not tested and may lead to unexpected behaviour.

Let's describe all possible scenarios, expected behaviour for each of them and extend test coverage.

Results.

Scenarios to check

Startup scenarios, when nodes start in different order to check grid assembles and operates correctly. Seems, it make sense when CMG node start first, because grid can't be assembled in otherwise.
Restart scenarios, when stop-then-start node of different roles on various grid configurations, and check services degradation/restoration. In contrast to startup scenarios, some grid configuration work different, e.g. grid without CMG node.
Stop scenarios are covered in "restart scenarios."

Operations

Operation in RO transaction. This requires at least one follower alive.
Operation in RW transaction. This requires quorum (leader).
Operation in implicit transaction.
Creating distribution zone. This requires MetaStorage quorum (+ maybe data nodes). Not implemented yet.
DDL operation. Create table in distribution zone, which is has quorum/followers without quorum/no followers. This requires Metastorage quorum.
Starting new node and check it present in logical and/or physical topologies. This requires CMG quorum (+maybe Metastorage).