[HADOOP-10129] Distcp may succeed when it fails

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
• Fix Version/s: 0.23.11, 2.3.0
• Component/s: tools/distcp
• Target Version/s: 0.23.11
• Hadoop Flags: Reviewed
• Assignee: Daryn Sharp
• Reporter: Daryn Sharp
• Created: 26/Nov/13 18:59
• Updated: 12/May/16 18:21
• Resolved: 05/Dec/13 15:55

Description

Distcp uses IOUtils.cleanup to close its output streams w/o previously attempting to close the streams. IOUtils.cleanup will swallow close or implicit flush on close exceptions. As a result, distcp may silently skip files when a partial file listing is generated, and/or appear to succeed when individual copies fail.