Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Upgrade the tool:
0. Check
0.1 Datanode service is shut down, check process name
0.2 Check the layout version of DN, whether Schema V3 is supported.
0.3 Check if volume rocksdb already exists, if yes, prompt the user whether to use this rocksdb directly as the storage of metadata after migrate, and prompt the user that the current rocksdb will be automatically backed up (print the backup file name and path), if no, end and exit.
0.4 If volume rocksdb does not exist, prompt the user to perform finalizae DN operation.
0.5 Check if there is a migrate completed file, if so, prompt the user that the volume has been migrated before, end exit.
0.6 Check whether the DN state is in the containcen state (DN state in the VERSION file), if not, prompt the user to set the DN to the maintenance state on the SCM side, end and exit.
1. Migrate data
Support two ways, one to migrate one volume at a time, and one to migrate all the volumes (multi-threaded, one thread for each volume).
Generate a LOCK file under each volume to be migrated this time, and exit the end command if the creation fails.
Iterate through each container under a volume (containers with all states)
1.1 If Schema V1, skip.
1.2 If it is Schema V2
a. Open container rocksdb and read all the key values.
b. Convert the key format to schema v3 format.
c. Write the new key and value to the volume rocksdb.
d. Backup the container yaml file
e. Modify the container yaml file with v3 in the version field, modify the metadata path, and recalculate the file checksume.
If the yaml file is missing, skip the container.
If checksdb is missing, modify the container yaml file directly.
Disk full, disk IO error, the end of this thread (if a single disk operation, the entire tool ends) - Documentation: We recommend that the user first solve the disk problem (disk full), prompted to delete the LOCK file under the volume, for the volume, re-run the command again
2. Choose a container ID at random, get the metadata of V2 and V3 respectively, and compare whether the metadata is consistent. If the metadata is consistent, the command ends successfully. If not, an error occurs, exit, and keep the site.
3. Delete the LOCK files under all designed volumes.
4. Generate an identifier file that identifies the completion of the migrate under the current volume current, and record the timestamp.
Attachments
Issue Links
- links to