[IMPALA-11276] Flaky TestIcebergTable.test_partitioned_insert

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Epic Color: ghx-label-4
• Assignee: Li Penglin
• Reporter: Tamas Mate
• Created: 29/Apr/22 13:44
• Updated: 23/Jun/22 12:41
• Resolved: 23/Jun/22 12:41

Description

TestIcebergTable.test_partitioned_insert became flaky, I assume it is related to 
IMPALA-11256 and the file descriptor sort.

01:37:12 [gw2] linux2 -- Python 2.7.16 /data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/bin/../infra/python/env-gcc7.5.0/bin/python
01:37:12 query_test/test_iceberg.py:87: in test_partitioned_insert
01:37:12     use_db=unique_database)
01:37:12 common/impala_test_suite.py:757: in run_test_case
01:37:12     self.__verify_results_and_errors(vector, test_section, result, use_db)
01:37:12 common/impala_test_suite.py:588: in __verify_results_and_errors
01:37:12     replace_filenames_with_placeholder)
01:37:12 common/test_result_verifier.py:469: in verify_raw_results
01:37:12     VERIFIER_MAP[verifier](expected, actual)
01:37:12 common/test_result_verifier.py:278: in verify_query_result_is_equal
01:37:12     assert expected_results == actual_results
01:37:12 E   assert Comparing QueryTestResults (expected vs actual):
01:37:12 E     row_regex:'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/[^=]*.0.parq','.*','' != 'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/d_year=2020/i=5/s_bucket=2/3a457bd9f7ad3bf4-1e4a0b0800000000_1286496050_data.0.parq','831B',''
01:37:12 E     row_regex:'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/d_year=2020/i=5/s_bucket=2/.*.0.parq','.*','' != 'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/d_year=2020/i=6/s_bucket=0/3a457bd9f7ad3bf4-1e4a0b0800000000_600591424_data.0.parq','831B',''
01:37:12 E     row_regex:'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/d_year=2020/i=6/s_bucket=0/.*.0.parq','.*','' != 'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/e746d921b277fd1d-2d81198f00000000_2048169195_data.0.parq','852B',''
01:37:12 E     row_regex:'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/i=3/d=2020-12-09/.*.0.parq','.*','' == 'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/i=3/d=2020-12-09/3042238130201abd-bb23fde000000000_992458870_data.0.parq','831B',''
01:37:12 E     row_regex:'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/i=4/d=2020-12-10/.*.0.parq','.*','' == 'hdfs://localhost:20500/test-warehouse/test_partitioned_insert_3c7f990.db/ice_alter_part/data/i=4/d=2020-12-10/3042238130201abd-bb23fde000000000_581725288_data.0.parq','838B',''

Issue Links

• relates to: IMPALA-11256 SHOW FILES on Iceberg tables lists all files in table directory