[HIVE-17970] MM LOAD DATA with OVERWRITE doesn't use base_n directory concept

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Labels: mm-gap-2
• Target Version/s: 3.0.0
• Assignee: Sergey Shelukhin Assign to me
• Reporter: Eugene Koifman
• Created: 02/Nov/17 17:17
• Updated: 22/May/18 23:14
• Resolved: 23/Apr/18 18:25

Description

Judging by

Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType, boolean isSrcLocal,
      boolean isSkewedStoreAsSubdir, boolean isAcid, boolean hasFollowingStatsTask,
      Long txnId, int stmtId, boolean isMmTable)

LOAD DATA with OVERWRITE will delete all existing data then write new data into the table. This logic makes sense for non-acid tables but for Acid/MM it should work like INSERT OVERWRITE statement and write new data to base_n/. This way the lock manager can be used to either get an X lock for IOW and thus block all readers or let it run with SemiShared and let readers continue and make the system more concurrent.

Issue Links

• is related to: HIVE-17856 MM tables - IOW is not ACID compliant
• is related to: HIVE-17361 Support LOAD DATA for transactional tables
• links to: RB