Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6422

Compute stats tablesample spends a lot of time in powf()

    XMLWordPrintableJSON

    Details

      Description

      Mostafa Mokhtar did perf profiling for COMPUTE STATS TABLESAMPLE and discovered that a lot of time is spent on finalizing HLL intermediates. Most time is spent in powf().

      Relevant snippet from AggregateFunctions::HllFinalEstimate() in aggregate-functions-ir.cc:

        for (int i = 0; i < num_buckets; ++i) {
          harmonic_mean += powf(2.0f, -buckets[i]);
          if (buckets[i] == 0) ++num_zero_registers;
        }
      

      Since we're doing a power of 2 using ldexp() should be much more efficient.

      I did a microbenchmark and found that ldexp() is >10x faster than powf() for this scenario.

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: