[IMPALA-11291] minidump-test is flaky

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.2.0
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Epic Color: ghx-label-3
• Assignee: Michael Smith
• Reporter: Quanlong Huang
• Created: 16/May/22 02:17
• Updated: 23/Aug/22 05:52
• Resolved: 17/May/22 16:23

Description

Minidump.DumpCallback and Minidump.DumpCallbackWithThread added by IMPALA-11275 are flaky.

Minidump.DumpCallback is found to fail in an unrelated build:
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/16449/testReport/junit/(root)/Minidump/DumpCallback/

/home/ubuntu/Impala/be/src/util/minidump-test.cc:47
Value of: lines.size()
  Actual: 4
Expected: 3

While looping the test locally, I can also find failures of DumpCallbackWithThread:

seed = 1652667209
Note: Google Test filter = Minidump.*
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from Minidump
[ RUN      ] Minidump.DumpCallback
[       OK ] Minidump.DumpCallback (1 ms)
[ RUN      ] Minidump.DumpCallbackWithThread
/home/quanlong/workspace/Impala/be/src/util/minidump-test.cc:76: Failure
Value of: lines.size()
  Actual: 4
Expected: 3
/home/quanlong/workspace/Impala/be/src/util/minidump-test.cc:77: Failure
Value of: boost::regex_match(lines[0], minidump_in_thread)
  Actual: false
Expected: true
/home/quanlong/workspace/Impala/be/src/util/minidump-test.cc:78: Failure
Value of: boost::regex_match(lines[1], wrote_minidump)
  Actual: false
Expected: true
/home/quanlong/workspace/Impala/be/src/util/minidump-test.cc:79: Failure
Value of: lines[2]
  Actual: "Wrote minidump to /tmp/arbitrary/path/b5cfb1b6-14cc-4a43-676f738b-52e07232.dmp"
Expected: ""
[  FAILED  ] Minidump.DumpCallbackWithThread (0 ms)
[----------] 2 tests from Minidump (1 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (1 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Minidump.DumpCallbackWithThread

CC MikaelSmith

Issue Links

• is caused by: IMPALA-11275 Dump thread debug information when crashing/generating a minidump
• links to: Code Review