Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5520

TopN node does not reuse string memory

    Details

      Description

      In some cases TopN will use excessive memory. E.g. if you have a large number of input rows containing strings sorted in reverse order, it will allocate memory for all of the strings and never free it.

      We should either recycle the allocations or periodically re-materialise and garbage collect the old allocations

      There is a TODO in the code already.

            // TODO: DeepCopy() will allocate new buffers for the string data. This needs
            // to be fixed to use a freelist
      

        Attachments

          Activity

            People

            • Assignee:
              bikramjeet.vig Bikramjeet Vig
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: