[HIVE-14776] Skip 'distcp' call when copying data from HDSF to S3

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: Hive
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 16/Sep/16 15:15
• Updated: 19/Sep/16 00:51

Description

Hive uses 'distcp' to copy files in parallel between HDFS encryption zones when the hive.exec.copyfile.maxsize threshold is lower than the file to copy. This 'distcp' is also executed when copying to S3, but it is causing slower copies.

We should not invoke distcp when copying to blobstore systems.