[HIVE-19279] remove magic directory skipping from CopyTask

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Transactions
• Assignee: Unassigned
• Reporter: Sergey Shelukhin
• Created: 23/Apr/18 21:22
• Updated: 27/Apr/18 23:57

Description

Follow up from HIVE-17657.
Code exists in copytask that copies files (fancy that); however, when listing the files, if a single directory exists at the source with no other files, it will skip the directory and copy the files inside instead.
This directory in various tests is either the "data" directory from export, or some random partition directory ("foo=bar") that if not skipped makes it into the real partition directory at the destination.
The directory is not skipped if it's not by itself, i.e. any other files or directories are present.

This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead of relying on this behavior.

Issue Links

• relates to: HIVE-17657 export/import for MM tables is broken