[IMPALA-4727] Sequence of DROP IF EXISTS + CREATE may fail if there is a concurrent DROP of the same catalog object.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.8.0
• Component/s: Catalog
• Labels: usability
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Alexander Behm
• Created: 05/Jan/17 00:17
• Updated: 25/Jan/17 23:22

Description

The following strange behavior is a result of the fix for IMPALA-3641. Please refer to IMPALA-3641 for an explanation of why this new bug here is preferred over IMPALA-3641.

The CREATE portion of a sequence of DROP IF EXISTS + CREATE from the same session may fail if there is a concurrent DROP [IF EXISTS] on the same object from a different session. Example and explanation:

• Thread A is executing a DROP TABLE t
• Thread B is executing a DROP TABLE IF EXISTS t
• Thread A drops the table first, gets the dropped object from the catalogd and applies the change to its local impalad (but not the impalad that thread B is connected to)
• Thread B executes DROP TABLE IF EXISTS t on the catalogd and finds that t is already gone, so an empty response is returned and no changes are applied to B's local impalad
• The impalad that B is connected to has not received a statestore update containing the DROP TABLE t executed by A
• B now executes CREATE TABLE t and finds that the table already exists

In general, a sequence of DROP+ADD from a single session is not guaranteed to always succeed if there are concurrent DROP/ADD operations of the same object from another session. However, this case is particularly strange because there is only a single ADD which was preceded by two DROPs, but the ADD still fails.