[HUDI-2477] Restore fails after adding rollback plan and rollback.requested instant w/ metadata enabled

• Type: Task
• Status: Resolved
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 0.11.0
• Component/s: writer-core
• Labels: pull-request-available
• Story Points: 1
• Epic Link: Metadata Table for File Listing & Query Planning
• Assignee: sivabalan narayanan
• Reporter: sivabalan narayanan
• Created: 21/Sep/21 20:20
• Updated: 11/Jan/22 01:16

Description

restore triggers rollback of N commits and then finally will commit the restore. None of rollbacks will be published to timeline. 

But after we have added the rollback.requested instant, restore is breaking w/ metadata enabled. 

Here is what is happening:

Restore

     schedule rollback for all of N commits. this will produce rollback.requested instants to timeline. Remember we can't skip this publishing, bcoz, rollback action executor depends on this. 

    trigger rollback action executor. which will execute the rollback. but this time we may not publish the rollbacks. and so there won't be a rollback completed instant. 

and now to finalize restore, we apply the changes to metadata table before we can commit the restore to datatable. Here is where the issue is. We do check if bootstrapping is required. chances that last synced instant to metadata table is not active anymore in data table and so it triggers a bootstrap. but we do allow bootstrap only if there are no pending operations in datatable. But all rollbacks are surfacing as pending operations and hence we fail here. 

 

This could also be an issue when we try to play with bootstrap in original dataset. 

bootstrap. and for some reason you want to rollback bootstrap. this might end up in this state too. 

to illustrate clearly. 

bootstrap

    also apply changes to metadata. there is only one commit.

rollback bootstrap

   this is a restore operation. so, we first do a rollback which will create rollback.requested instant. 

               and to finalize restore, we try to apply the restore to metadata. 

                    this goes into bootstrap code path. last synced instant is not found in datatimeline. we assume its archived and so trigger a rebootstrap and so delete the metadata table. 

                    and then try to do the actual bootstrap. but since there is a pending operation in datatimeline (rollback.requested), we will not do any bootstrap only. and so the state remains. i.e. metadata table is deleted. and the actually applying restore commit will fail. 

 

 

We also need to think even if metadata is enabled, should we leave the rollback instant in timeline itself.or should we clean it up after committing restore to timeline. 

 

 

 

Issue Links

• links to: GitHub Pull Request #4518