[HADOOP-15789] DistCp does not clean staging folder if class extends DistCp

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: tools/distcp
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Lawrence Andrews
• Reporter: Lawrence Andrews
• Created: 20/Jul/17 19:38
• Updated: 08/Jul/22 11:35
• Resolved: 08/Jul/22 11:35

Description

My code extends Distcp class and for some reason if distcp fails staging folder is not delete and this staging folder piles up occupying space on hdfs.

After checking the code i found that cleanup() function is private. Making the cleanup() method as public, user should be able to invoke the cleanup if job fails.

This works fine with command line argument. But fails only if we extend Distcp class.

Issue Links

• is depended upon by: HADOOP-15788 Improve Distcp for long-haul/cloud deployments
• links to: GitHub Pull Request #4534