[HIVE-24852] Add support for Snapshots during external table replication

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Ayush Saxena
• Reporter: Ayush Saxena
• Created: 07/Mar/21 19:09
• Updated: 27/Jul/23 06:21
• Resolved: 18/May/21 16:17

Description

Add support for use of snapshot diff for external table replication.

Issue Links

• is related to: HDFS-15916 DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff
• is related to: HIVE-26033 Repl Load fails with Wrong FS error.
• is related to: HADOOP-17531 DistCp: Reduce memory usage on copying huge directories
• is related to: HADOOP-17620 DistCp: Use Iterator for listing target directory as well
• is related to: HIVE-24750 Create a single copy task for external tables within default DB location
• is related to: HIVE-25035 Allow creating single copy tasks for configured paths during external table replication
• is related to: HIVE-25133 Allow custom configs for database level paths in external table replication
• links to: GitHub Pull Request #2043