Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22103

Move HashAggregateExec parent consume to a separate function in codegen

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.3.0
    • SQL
    • None

    Description

      HashAggregateExec codegen uses two paths for fast hash table and a generic one.
      It generates code paths for iterating over both, and both code paths generate the consume code of the parent operator, resulting in that code being expanded twice.
      This leads to a long generated function that might be an issue for the compiler (see e.g. SPARK-21603).
      I propose to remove the double expansion by generating the consume code in a helper function that can just be called from both iterating loop.

      Attachments

        Activity

          People

            juliuszsompolski Juliusz Sompolski
            juliuszsompolski Juliusz Sompolski
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: