[HIVE-17214] check/fix conversion of unbucketed non-acid to acid

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 31/Jul/17 17:38
• Updated: 22/May/18 23:59
• Resolved: 16/Oct/17 18:59

Description

bucketed tables have stricter rules for file layout on disk - bucket files are direct children of a partition directory.

for un-bucketed tables I'm not sure there are any rules
for example, CTAS with Tez + Union operator creates 1 directory for each leg of the union

Supposedly Hive can read table by picking all files recursively.
Can it also write (other than CTAS example above) arbitrarily?
Does it mean Acid write can also write anywhere?

Figure out what can be supported and how can existing layout can be checked? Examining a full "ls -l -R" for a large table could be expensive.

Issue Links

• is blocked by: HIVE-17505 hive.optimize.union.remove=true doesn't work with insert into