[MAPREDUCE-5075] DistCp leaks input file handles

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 0.23.7, 2.1.0-beta
• Component/s: distcp
• Hadoop Flags: Reviewed
• Assignee: Chris Nauroth
• Reporter: Chris Nauroth
• Created: 15/Mar/13 21:55
• Updated: 12/May/16 18:22
• Resolved: 20/Mar/13 10:45

Description

DistCp wraps the InputStream for each input file it reads in an instance of ThrottledInputStream. This class does not close the wrapped InputStream. RetriableFileCopyCommand guarantees that the ThrottledInputStream gets closed, but without closing the underlying wrapped stream, it still leaks a file handle.

Issue Links

• is duplicated by: HDFS-4601 DistCp's ThrottledInputStream leaks.
• relates to: MAPREDUCE-4401 Enhancements to MapReduce for Windows Server and Windows Azure development and runtime environments