[HDFS-951] DFSClient should handle all nodes in a pipeline failed.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Assignee: Unassigned
• Reporter: He Yongqiang
• Created: 05/Feb/10 02:48
• Updated: 03/Apr/14 02:40
• Resolved: 03/Apr/14 02:40

Description

processDatanodeError-> setupPipelineForAppendOrRecovery will set streamerClosed to be true if all nodes in the pipeline failed in the past, and just return.
Back to run() in data streammer, the logic
if (streamerClosed || hasError || dataQueue.size() == 0 || !clientRunning)

{ continue; }

will just let set closed=true in closeInternal().

And DataOutputStream will not get a chance to clean up. The DataOutputStream will throw exception or return null for following write/close.
It will leave the file in writing in incomplete state.

Issue Links

• is related to: HDFS-278 Should DFS outputstream's close wait forever?