[IMPALA-4631] plan-fragment-executor.cc:518] Check failed: other_time <= total_time (25999394 vs. 25999393)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: flaky
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Daniel Hecht
• Created: 08/Dec/16 23:29
• Updated: 21/Feb/20 06:34
• Resolved: 21/Feb/20 01:07

Description

This dcheck occasionally fires:

impalad.FATAL:F1201 22:35:58.617157 30293 plan-fragment-executor.cc:518] Check failed: other_time <= total_time (25999394 vs. 25999393)

I suspect the problem is with using floating point operations in places like this:

   timespec ts;
    clock_gettime(OsInfo::fast_clock(), &ts);
    return ts.tv_sec * 1e9 + ts.tv_nsec;

and because floating point doesn't distribute, and we can end up with

 c * (a + b) < c * a + c * b 

which is effectively what the dcheck does.

Issue Links

• is duplicated by: IMPALA-9407 Impalad crash in RuntimeProfile::EventSequence
• is duplicated by: IMPALA-4717 Impalad crash during TestRefreshPartition::test_confirm_individual_refresh
• is duplicated by: IMPALA-6316 impalad crashes after hadoopZeroCopyRead failure
• is duplicated by: IMPALA-5639 Exhaustive tests hitting DCHECK in FragmentInstanceState::Close()
• is related to: IMPALA-6450 Hit DCHECK in RuntimeProfile::EventSequence::Start()