Considering the following code:
This logs the following exception:
I am not a spark expert but after investigation, I realized that the generated doConsume method is responsible of the exception.
Indeed, avg calls several times org.apache.spark.sql.execution.CodegenSupport.constructDoConsumeFunction. The 1st time with the 'avg' Expr and a second time for the base aggregation Expr (count and sum).
The problem comes from the generation of parameters in CodeGenerator:
The freshNameIds already contains agg_expr_[1..6] from the 1st call.
The second call is made with agg_expr_[1..12] and generates the following names:
agg_expr_[11|21|31|41|51|61|11|12]. We then have a parameter name conflicts in the generated code: agg_expr_11.
Appending the 'id' in s"$fullName$id" to generate unique term name is source of conflict. Maybe simply using undersoce can solve this issue : $fullName_$id"