[GOBBLIN-2072] Permission mismatch in manifest distcp where empty folders are copied

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: gobblin-core
• Assignee: Abhishek Tiwari
• Reporter: William Lo
• Created: 28/May/24 20:23
• Updated: 05/Jun/24 18:47

Description

In Manifest distcp, empty folders are being set with the incorrect permissions in SetPermissionStep after being correctly set in the CopyDataPublisher.

The suspected reason is that the folders themselves are being treated as ancestors, and this causes the permission checking of the source and destination ancestors to act incorrectly. TODO: We should consolidate our permission comparison and logic given that there are a multitude of different implementations and they each seem to have their own quirks.

Correct CopyDataPublisher:

Setting destination directory hdfs://cluster/a/b/c owner and permission to rwxr-x---

Incorrect permission commit step:

[Azkaban] Setting permission rwxr-xr-x on path /a/b/c