[IMPALA-11352] CI: Transient failure in test_event_based_replication

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 4.1.0
• Component/s: Distributed Exec
• Epic Color: ghx-label-13
• Assignee: Michael Smith
• Reporter: Michael Smith
• Created: 13/Jun/22 21:04
• Updated: 15/Mar/23 23:47

Description

Transient failure in metadata.test_event_processing.TestEventProcessing.test_event_based_replication
during https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5815/.

Test output summary shows

metadata/test_event_processing.py:183: in test_event_based_replication
    self.__run_event_based_replication_tests()
metadata/test_event_processing.py:302: in __run_event_based_replication_tests
    EventProcessorUtils.wait_for_event_processing(self)
util/event_processor_utils.py:70: in wait_for_event_processing
    allow_greater=True)
common/impala_service.py:143: in wait_for_metric_value
    self.__metric_timeout_assert(metric_name, expected_value, timeout)
common/impala_service.py:210: in __metric_timeout_assert
    assert 0, assert_string
E   AssertionError: Metric catalog.curr-version did not reach value 9943 in 10s.

Logs attached. Initial guess is something in parallel execution is delaying catalog updates. I see messages around 00:38:17 for catalog version 9938 (which is what the captured metrics show) and catalog version 9961 at 0:38:19. Test bumps are labeled with 0:38:21.

Issue Links

• relates to: IMPALA-11968 test_event_based_replication fails with timeout
• relates to: IMPALA-12002 COMMIT_COMPACTION_EVENT are not correctly skipped if the db not found