Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10374

Limit page iteration at BufferedTupleStream::DebugString()

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Impala 3.4.0
    • Fix Version/s: Impala 4.0.0
    • Component/s: Backend
    • Labels:
    • Epic Color:
      ghx-label-4

      Description

      BufferedTupleStream::DebugString() iterate std::list<Page> that can potentially grow very large. While most references to this method are behind DCHEK logging (thus stripped in release build), one of them is used to build Status message through call to GroupingAggregator::Partition::DebugString() at this line:

      https://github.com/apache/impala/blob/5530b62/be/src/exec/grouping-aggregator-partition.cc#L155

      IMPALA-9851 already cap error message to 128kb at most, but that does not eliminate the cost to iterate page list in BufferedTupleStream::DebugString().

      We should limit number of page being iterated in this method to avoid unnecessary long string operation.

        Attachments

          Activity

            People

            • Assignee:
              rizaon Riza Suminto
              Reporter:
              rizaon Riza Suminto
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: