[HADOOP-17628] Distcp contract test is really slow with ABFS and S3A; timing out

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.3.2
• Component/s: fs/azure, fs/s3, test, tools/distcp
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Assignee: Steve Loughran
• Reporter: Bilahari T H
• Created: 09/Apr/21 02:35
• Updated: 09/Nov/21 13:45
• Resolved: 02/Aug/21 12:33

Description

The test case testDistCpWithIterator in AbstractContractDistCpTest is consistently timing out.

 

Issue Links

• is caused by: HADOOP-17531 DistCp: Reduce memory usage on copying huge directories
• is caused by: HADOOP-17620 DistCp: Use Iterator for listing target directory as well
• links to: GitHub Pull Request #3240