[HADOOP-18096] Distcp: Sync moves filtered file to home directory rather than deleting

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.2
• Fix Version/s: 3.4.0, 3.3.2
• Component/s: tools/distcp
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.2
• Hadoop Flags: Reviewed
• Assignee: Ayush Saxena
• Reporter: Ayush Saxena
• Created: 27/Jan/22 18:21
• Updated: 11/Feb/24 23:32
• Resolved: 10/Feb/22 20:59

Description

Distcp sync with snapshot, if the file being copied is renamed to a path which is in the exclusion filter, tries to delete the file.

But instead of deleting, it moves the file to home directory

 

Issue Links

• is broken by: HDFS-14869 Data loss in case of distcp using snapshot diff. Replication should include rename records if file was skipped in the previous iteration
• links to: GitHub Pull Request #3940