[HIVE-21269] Mandate -update and -delete as DistCp options to sync data files for external tables replication.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: repl
• Labels: DR pull-request-available replication
• Target Version/s: 4.0.0
• Assignee: Sankar Hariappan
• Reporter: Sankar Hariappan
• Created: 14/Feb/19 06:54
• Updated: 17/Nov/22 08:54
• Resolved: 15/Feb/19 04:29

Description

Currently, external tables replication, copies the data in directory level. So, if target directory exist, then DistCp should compare and update or skip data files in the directory instead of creating new directory inside pre-existing target directory.
This can be achieved using -update.
Also, -delete option is needed to delete the files missing in source directory but present in target.
Hive should mandate these DistCp options even if user passes other options.

Issue Links

• links to: GitHub Pull Request #535