[IMPALA-11072] TestSpillingDebugActionDimensions.test_spilling is flaky

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.1.0
• Component/s: Infrastructure
• Epic Color: ghx-label-8
• Assignee: Riza Suminto
• Reporter: Riza Suminto
• Created: 10/Jan/22 17:30
• Updated: 19/Feb/22 18:31
• Resolved: 13/Jan/22 19:59

Description

We have seen some failure of TestSpillingDebugActionDimensions.test_spilling in GVO jenkins job and downstream nightly tests. Latest one happen in https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/15503/

 

 

query_test/test_spilling.py:75: in test_spilling
    self.run_test_case('QueryTest/spilling', vector)
common/impala_test_suite.py:743: in run_test_case
    update_section=pytest.config.option.update_results)
common/test_result_verifier.py:636: in verify_runtime_profile
    actual))
E   AssertionError: Did not find matches for lines in runtime profile:
E   EXPECTED LINES:
E   row_regex: .*SpilledPartitions: .* \([1-9][0-9]*\)
E   
E   ACTUAL PROFILE:
E   Query (id=8b433ac02c4d3fd2:3c50b7c400000000):
E      - InactiveTotalTime: 0.000ns
E      - TotalTime: 0.000ns
E     Summary:
E       Session ID: 9448bded8acf05c6:428a7a797f6b9483
E       Session Type: BEESWAX
E       Start Time: 2022-01-08 09:37:07.647285000
E       End Time: 2022-01-08 09:37:15.514936000
E       Query Type: QUERY
E       Query State: FINISHED
E       Impala Query State: FINISHED
E       Query Status: OK
E       Impala Version: impalad version 4.1.0-SNAPSHOT RELEASE (build 560ff976d3a08920a08b4ce3325a1dd9dbe81765)
E       User: ubuntu
E       Connected User: ubuntu
E       Delegated User: 
E       Network Address: ::ffff:127.0.0.1:44648
E       Default Db: tpch_parquet
E       Sql Statement: select count(l1.l_tax)
E   from
E   lineitem l1,
E   lineitem l2,
E   lineitem l3
E   where
E   l1.l_tax < 0.01 and
E   l2.l_tax < 0.04 and
E   l1.l_orderkey = l2.l_orderkey and
E   l1.l_orderkey = l3.l_orderkey and
E   l1.l_comment = l3.l_comment and
E   l1.l_shipdate = l3.l_shipdate
E       Coordinator: ip-172-31-21-231:27000
E       Query Options (set by configuration): BUFFER_POOL_LIMIT=225443840,MT_DOP=0,DEFAULT_SPILLABLE_BUFFER_SIZE=262144,TIMEZONE=Universal,CLIENT_IDENTIFIER=query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[protocol:beeswax|exec_option:{'mt_dop':0;'debug_action':None;'default_spillable_buffer_size':'256k'}|table_format:parquet/none]
E       Query Options (set by configuration and planner): BUFFER_POOL_LIMIT=225443840,MT_DOP=0,DEFAULT_SPILLABLE_BUFFER_SIZE=262144,TIMEZONE=Universal,CLIENT_IDENTIFIER=query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[protocol:beeswax|exec_option:{'mt_dop':0;'debug_action':None;'default_spillable_buffer_size':'256k'}|table_format:parquet/none],MINMAX_FILTER_THRESHOLD=0.5,MINMAX_FILTERING_LEVEL=PAGE 
...

 

We should lower the configured BUFFER_POOL_LIMIT for this test to less than 215MB so that it spill more consistently.

 

Issue Links

• relates to: IMPALA-11135 TestSpillingDebugActionDimensions.test_spilling seems to be flaky