Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22103

Move HashAggregateExec parent consume to a separate function in codegen

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None

      Description

      HashAggregateExec codegen uses two paths for fast hash table and a generic one.
      It generates code paths for iterating over both, and both code paths generate the consume code of the parent operator, resulting in that code being expanded twice.
      This leads to a long generated function that might be an issue for the compiler (see e.g. SPARK-21603).
      I propose to remove the double expansion by generating the consume code in a helper function that can just be called from both iterating loop.

        Attachments

          Activity

            People

            • Assignee:
              juliuszsompolski Juliusz Sompolski
              Reporter:
              juliuszsompolski Juliusz Sompolski
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: