[IMPALA-11671] run-all-tests.sh with Ozone fails listing DFS files

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.2.0
• Fix Version/s: Impala 4.2.0
• Labels: ci-failure
• Epic Color: ghx-label-14
• Assignee: Michael Smith
• Reporter: Michael Smith
• Created: 18/Oct/22 00:05
• Updated: 31/Oct/22 04:48
• Resolved: 20/Oct/22 17:27

Description

hdfs dfs -ls -R ${FILESYSTEM_PREFIX}/test-warehouse \
      > ${IMPALA_LOGS_DIR}/file-list-end-${i}.log 2>&1

in run-all-tests.sh fails when testing with Ozone. Errors like

ls: Unable to get file status: volume: impala bucket: test-warehouse key: .Trash/jenkins/Current/impala/test-warehouse/test_insert_select_exprs_5162abf6.db/overflowed_decimal_tbl_2/_impala_insert_staging/c94353a630109577_202e902000000000
ls: Unable to get file status: volume: impala bucket: test-warehouse key: .Trash/jenkins/Current/impala/test-warehouse/tpch.db/ctas_cancel/_impala_insert_staging/454c6614d6c20d17_94fac58c00000000
ls: Unable to get file status: volume: impala bucket: test-warehouse key: .Trash/jenkins/Current/impala/test-warehouse/tpch.db/ctas_cancel1665933223846/_impala_insert_staging/7a46352d463f457e_fcabb6e000000000
ls: Unable to get file status: volume: impala bucket: test-warehouse key: .Trash/jenkins/Current/impala/test-warehouse/tpch.db/ctas_cancel1665933261855/_impala_insert_staging/d24d1c35c2a9360c_90e32ce200000000

are visible in file-list-end-1.log. Current theory is that Ozone is cleaning up trash files while we list them.

Issue Links

• is caused by: HDDS-4974 Intermittent error in Trash: unable to get file status