Details
-
Task
-
Status: Closed
-
Major
-
Resolution: Invalid
-
None
-
None
Description
I don't fully get this point. will follow up with Vinoth on the exact scenario.
here is my understanding: of a scenario using cloud stores that does not support append.
If there was crash during a commit, when listing log files to be logged, the last one which got crashed may not be part of the rollback plan. but thats should be fine. anyways, its not available via listing. and so I assume even during compaction those will not be available. we will proceed on with rollback by adding another log file. and this will get replayed to metadata table.
If you are talking about the case, where a crash happens when rollback itself is being logged and crashed just before committing to metadata table.
we should be ok here too. we will retry the rollback which will redo the action phase. and will add new log blocks (with same old logs that were part of failed writes, just that it may not be able to successfully delete). and this will get applied to metadata table. We just have to ensure when applying changes to metadata table, we consider all files from the plan and not just the ones that got successfully deleted.
- with hdfs type of cloud stores, where appends are allowed, we just create new log blocks. and hence should not be an issue.