[GOBBLIN-1973] Manifest file copy ignores copying files when source file permissions differ from dest

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-core
• Assignee: Abhishek Tiwari
• Reporter: William Lo
• Created: 12/Dec/23 22:04
• Updated: 14/Dec/23 21:35

Description

When Manifest distcp fails to set permissions for any reason, we would expect a retry of the copy to allow the datasets to reach eventual consistency. However, the current logic of checking if a file is eligible for copy is to check if the source file is newer, if so copy, and if the source file is exactly the same modification time as the destination file, also copy if the permissions are different. However, in most cases the destination will have a newer modification time once it is written and set permissions, so the source file will generally be older but we still want the permissions to match.