[HDFS-11032] [SPS]: Handling of block movement failure at the coordinator datanode

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: HDFS-10285, 3.2.0
• Component/s: datanode
• Hadoop Flags: Reviewed
• Assignee: Rakesh Radhakrishnan
• Reporter: Rakesh Radhakrishnan
• Created: 19/Oct/16 07:33
• Updated: 13/Aug/18 05:39
• Resolved: 23/Dec/16 01:08

Description

The idea of this jira is to discuss and implement an efficient failure(block movement failure) handling logic at the datanode cooridnator. Code reference.

Following are the possible errors during block movement:

1. Block pinned - no retries marked as success/no-retry to NN. It is not possible to relocate this block to another datanode.
2. Network errors(IOException) - no retries maked as failure/retry to NN.
3. No disk space(IOException) - no retries maked as failure/retry to NN.
4. Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a case that the file might have re-opened.

Issue Links

• requires: HDFS-11164 Mover should avoid unnecessary retries if the block is pinned