Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8737

Patch gperftools to fix O(n) scaling in PageHeap::AllocLarge()

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 3.3.0
    • Impala 4.0.0
    • Backend
    • None
    • ghx-label-2

    Description

      TCMalloc's PageHeap::AllocLarge() has O( n ) behavior. As the heap gets fragmented, this O( n ) behavior can lead to contention, because the thread executing PageHeap::AllocLarge() is holding a lock. In recent versions of gperftools, this code has been modified to have O(log n) behavior. This could reduce contention significantly in some cases.

      We can get this fix by using a more modern version of gperftools (see https://issues.apache.org/jira/browse/IMPALA-6784 ). However, the patches for the O(log n) behavior are fairly contained. Here are the two patches needed:

      https://github.com/gperftools/gperftools/commit/06c9414ec423ffe442c047b2560555f9d5847b1d

      https://github.com/gperftools/gperftools/commit/f1d3fe4a21e339a3fd6e4592ee7444484a7b92dc

      These would be easy to port to gperftools-2.5. This Jira tracks that effort (which is separate and would be superseded if we upgrade gperftools). 

      Attachments

        Issue Links

          Activity

            People

              joemcdonnell Joe McDonnell
              joemcdonnell Joe McDonnell
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: