[HADOOP-18242] ABFS Rename Failure when tracking metadata is in incomplete state

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.5
• Component/s: fs/azure
• Labels: pull-request-available
• Assignee: Mehakmeet Singh
• Reporter: Mehakmeet Singh
• Created: 18/May/22 04:02
• Updated: 21/Sep/22 10:16
• Resolved: 21/Sep/22 10:16

Description

If a node in the datacenter crashes while processing an operation, occasionally it can leave the Storage-internal blob tracking metadata in an incomplete state.  We expect this to happen occasionally, and so all API’s are designed in such a way that if this incomplete state is observed on a blob, the situation is resolved before the current operation proceeds.  However, this incident has exposed a bug specifically with the Rename API, where the incomplete state fails to resolve, leading to this incorrect failure.  As a temporary mitigation, if any other operation is performed on this blob – GetBlobProperties, GetBlob, GetFileProperties, SetFileProperties, etc – it should resolve the incomplete state, and rename will no longer hit this issue.

StackTrace:

2022-03-22 17:52:19,789 DEBUG [regionserver/euwukwlss-hg50:16020.logRoller] services.AbfsClient: HttpRequest: 404,RenameDestinationParentPathNotFound,cid=ef5cbf0f-5d4a-4630-8a59-3d559077fc24,rid=35fef164-101f-000b-1b15-3ed818000000,sent=0,recv=212,PUT,https://euwqdaotdfdls03.dfs.core.windows.net/eykbssc/apps/hbase/data/oldWALs/euwukwlss-hg50.tdf.qa%252C16020%252C1647949929877.1647967939315?timeout=90   

Issue Links

• links to: GitHub Pull Request #4331
• links to: GitHub Pull Request #4517