Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-13400

Incomplete ResourceProfile at multi-class AggregationNode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • Impala 4.4.0
    • None
    • Frontend
    • None

    Description

      AggregationNode.computeAggClassResourceProfile() call setSpillableBufferBytes and setMaxRowBufferBytes. However, ResourceProfile for each aggregation class are merged using ResourceProfile.sum() which waive setting both fields.

      https://github.com/apache/impala/blob/7167f3b4f0b6940f36705c95b8da17941557f721/fe/src/main/java/org/apache/impala/planner/AggregationNode.java#L605 

      https://github.com/apache/impala/blob/7167f3b4f0b6940f36705c95b8da17941557f721/fe/src/main/java/org/apache/impala/planner/ResourceProfile.java#L151 

      It should use ResourceProfile.combine() that will pick max 
      SpillableBufferBytes and max MaxRowBufferBytes.

      Attachments

        Activity

          People

            rizaon Riza Suminto
            rizaon Riza Suminto
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: