[MESOS-3423] Perf event isolator stops performing sampling if a single timeout occurs.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.24.0
• Fix Version/s: 0.25.0
• Component/s: agent, containerization
• Labels: twitter
• Target Version/s: 0.25.0
• Sprint: Twitter Mesos Q3 Sprint 5
• Story Points: 3
• Assignee: Benjamin Mahler
• Reporter: Vinod Kone
• Shepherd: Jie Yu
• Created: 14/Sep/15 18:29
• Updated: 26/Apr/17 16:54
• Resolved: 15/Sep/15 19:07

Description

Currently the perf event isolator times out a sample after a fixed extra time of 2 seconds on top of the sample time elapses:

    Duration timeout = flags.perf_duration + Seconds(2);

This should be based on the reap interval maximum.

Also, the code stops sampling altogether when a single timeout occurs. We've observed time outs during normal operation, so it would be better for the isolator to continue performing perf sampling in the case of timeouts. It may also make sense to continue sampling in the case of errors, since these may be transient.