[IMPALA-10886] TestReusePartitionMetadata.test_reuse_partition_meta fails

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.1.0
• Fix Version/s: Impala 4.1.0
• Component/s: Catalog
• Labels: broken-build
• Epic Color: ghx-label-7
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 26/Aug/21 14:36
• Updated: 04/Dec/21 08:28
• Resolved: 04/Dec/21 08:26

Description

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/14670/testReport/junit/custom_cluster.test_local_catalog/TestReusePartitionMetadata/test_reuse_partition_meta/

custom_cluster/test_local_catalog.py:586: in test_reuse_partition_meta
    self.check_missing_partitions(unique_database, 1)
custom_cluster/test_local_catalog.py:595: in check_missing_partitions
    assert match.group(1) == str(partition_misses)
E   assert '0' == '1'
E     - 0
E     + 1

RCA

There is a bug in CatalogOpExecutor#updateCatalog() that we loss the createEventId of the new partitions. It should be used in the final call
of loadTableMetadata(). This bug causes intermittent failures in TestReusePartitionMetadata.test_reuse_partition_meta. The last two DMLs
of the test is dropping a partition and then creating it back by an INSERT. If the DROP_PARTITION event is processed after the INSERT
finishes, the partition will be dropped incorrectly. See more in the commit message of the fix.

Issue Links

• is caused by: IMPALA-10502 delayed 'Invalidated objects in cache' cause 'Table already exists'
• links to: Code Review