[IMPALA-6441] concurrent select binary search explain string pattern match is wrong

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 3.0
• Component/s: Infrastructure
• Target Version: Impala 3.0
• Epic Color: ghx-label-6
• Assignee: Michael Brown
• Reporter: Michael Brown
• Created: 25/Jan/18 01:59
• Updated: 27/Mar/18 18:47
• Resolved: 01/Feb/18 19:54

Description

The pattern match for finding a start point in the stress test's binary search was based on the explain output, and after memory reservations, was never updated. It had a silent failsafe path and chose the memory limit of impalad as a start point instead. This means the binary search, while known to take a long time, is certainly taking longer than it should.

The current pattern is:

MEM_ESTIMATE_PATTERN = re.compile(r"Estimated.*Memory=(\d+.?\d*)(T|G|M|K)?B")

tarmstrong can you confirm we need to match on the second line here?

Max Per-Host Resource Reservation: Memory=156.88MB
Per-Host Resource Estimates: Memory=14.67GB

Issue Links

• breaks: IMPALA-6447 Failure in tests.stress.concurrent_select