[IMPALA-10502] delayed 'Invalidated objects in cache' cause 'Table already exists'

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.4.0
• Fix Version/s: Impala 4.1.0
• Component/s: Catalog, Clients, Frontend
• Epic Color: ghx-label-13
• Assignee: Vihang Karajgaonkar
• Reporter: Adriano
• Created: 12/Feb/21 09:40
• Updated: 01/Feb/24 10:33
• Resolved: 13/Jul/21 22:13

Description

In fast paced environment where the interval between the step 1 and 2 is # < 100ms (a simplified pipeline looks like):

0- catalog 'on demand' in use and disableHmsSync (enabled or disabled: no difference)
1- open session to coord A -> DROP TABLE X -> close session
2- open session to coord A > CREATE TABLE X> close session

Results: the step 2 can fail with table already exist.

During the internal investigation was discovered that IMPALA-9913 will regress the issue in almost all scenarios.
However considering that the investigation are internally ongoing it is nice to have the event tracked also here.
Once we are sure that IMPALA-9913 fix these events we can close this as duplicate, in alternative carry on the investigation.

Issue Links

• causes: IMPALA-10886 TestReusePartitionMetadata.test_reuse_partition_meta fails
• fixes: IMPALA-10663 Coordinator might observe stale metadata in local catalog mode
• is duplicated by: IMPALA-10468 DROP events which are generated while a batch is being processed may add table incorrectly