[IMPALA-6422] Compute stats tablesample spends a lot of time in powf()

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.11.0
• Fix Version/s: Impala 2.1.2
• Component/s: Backend
• Labels: compute-stats performance
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-3
• Assignee: Alexander Behm
• Reporter: Alexander Behm
• Created: 19/Jan/18 04:46
• Updated: 07/Feb/23 11:14
• Resolved: 19/Jan/18 23:45

Description

mmokhtar did perf profiling for COMPUTE STATS TABLESAMPLE and discovered that a lot of time is spent on finalizing HLL intermediates. Most time is spent in powf().

Relevant snippet from AggregateFunctions::HllFinalEstimate() in aggregate-functions-ir.cc:

  for (int i = 0; i < num_buckets; ++i) {
    harmonic_mean += powf(2.0f, -buckets[i]);
    if (buckets[i] == 0) ++num_zero_registers;
  }

Since we're doing a power of 2 using ldexp() should be much more efficient.

I did a microbenchmark and found that ldexp() is >10x faster than powf() for this scenario.