[HDFS-10314] A new tool to sync current HDFS view to specified snapshot

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: tools
• Assignee: Yongjun Zhang
• Reporter: Yongjun Zhang
• Created: 19/Apr/16 21:54
• Updated: 03/Oct/16 20:43

Description

HDFS-9820 proposed adding -rdiff switch to distcp, as a reversed operation of -diff switch.

Upon discussion with Jing Zhao, we will introduce a new tool that wraps around distcp to achieve the same purpose.

I'm thinking about calling the new tool "rsync", similar to unix/linux command "rsync". The "r" here means remote.

The syntax that simulate -rdiff behavior proposed in HDFS-9820 is

rsync <fromSnapshotName>  <toSnapshotName>  <source> <target>

This command ensure <fromSnapshotName> is newer than <toSnapshotName>.

I think, In the future, we can add another command to have the functionality of -diff switch of distcp.

sync <fromSnapshotName>  <toSnapshotName>  <source> <target>

that ensures <fromSnapshotName> is older than <toSnapshotName>.

Thanks Jing Zhao.

Issue Links

• relates to: HDFS-9820 Improve distcp to support efficient restore to an earlier snapshot