Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5481

RowDescriptors should be shared, rather than copied

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.10.0
    • Impala 2.10.0
    • None
    • None
    • ghx-label-8

    Description

      One of the RowBatch c'tors copies the row descriptor into the row batch. This leads to a lot of allocation churn since RowDescriptor contains some vector members, and since the descriptor is usually the same the copies are unnecessary.

      Instead, we should consider allocating the RowDescriptor once from an object pool, and sharing it amongst all row batches that need that descriptor.

      In some tests, RowDescriptor() shows up as 20% of the tcmalloc allocation time.

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue IMPALA-5528 tcmalloc contention much higher with concurrency after KRPC patch

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-2567 KRPC milestone 1

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              henryr Henry Robinson
              henryr Henry Robinson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: