[IMPALA-3663] Improve path matching in ERRORS section of test files

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.6.0
• Component/s: Infrastructure
• Labels: ramp-up test
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Skye Wanderman-Milne
• Created: 02/Jun/16 22:41
• Updated: 04/Jan/17 23:58

Description

We should implement more general-purpose file path matching in test_result_verifier.py, specifically to improve this logic: https://github.com/cloudera/Impala/blob/cdh5-trunk/tests/common/test_result_verifier.py#L246. The current logic appears very specifically tailored to certain error message formats. Combined with IMPALA-1872, this makes it impossible to capture some error messages (e.g., "File 'hdfs://localhost:20500/test-warehouse/bad_avro_snap_avro_snap/truncated_string.avro' is corrupt: truncated data block at offset 195").

For example, one possible solution would be introducing wildcards, e.g. "File $WILDCARD/bad_avro_snap_avro_snap/truncated_string.avro is corrupt".

Changing this logic may require also modifying current test files that use the "file: hdfs://regex.$" pattern in their ERRORS sections.

Issue Links

• relates to: IMPALA-1872 test_result_verifier error msg normalization should not rely on hdfs qualified paths