[HIVE-19739] Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0, 4.0.0-alpha-1
• Component/s: HiveServer2, repl
• Labels: DR pull-request-available replication
• Target Version/s: 3.1.0, 4.0.0
• Assignee: Sankar Hariappan
• Reporter: Sankar Hariappan
• Created: 30/May/18 12:39
• Updated: 21/Oct/22 06:59
• Resolved: 17/Jun/18 06:43

Description

Currently. bootstrap REPL LOAD have added checkpoint identifiers in DB/table/partition object properties once the data/metadata related to the object is successfully loaded.

If the Db exist and is not empty, then currently we are throwing exception. But need to support it for the retry scenario after a failure.

If there is a retry of bootstrap load using the same dump, then instead of throwing error, we should check if any of the tables/partitions are completely loaded using the checkpoint identifiers. If yes, then skip it or else drop/create them again.

If the bootstrap load is performed using different dump, then it should throw exception.

Allow bootstrap on empty Db only if ckpt property is not set. Also, if bootstrap load is completed on the target Db, then shouldn't allow bootstrap retry at all.

Issue Links

• is related to: HIVE-19499 Bootstrap REPL LOAD shall add tasks to create checkpoints for db/tables/partitions.
• links to: GitHub Pull Request #366