[SPARK-39525] AdaptiveQueryExecSuite fails with Scala 2.13

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Component/s: SQL
• Assignee: Bruce Robbins
• Reporter: Hyukjin Kwon
• Created: 20/Jun/22 01:43
• Updated: 12/Dec/22 18:10
• Resolved: 20/Jun/22 15:35

Description

https://github.com/apache/spark/runs/6912247429?check_suite_focus=true

2022-06-16T05:29:18.0189547Z [info] - SPARK-36020: Check logical link in remove redundant projects *** FAILED *** (33 milliseconds)
2022-06-16T05:29:18.0663300Z [info]   java.lang.RuntimeException: Once strategy's idempotence is broken for batch Infer Filters
2022-06-16T05:29:18.0665308Z [info]  Project [key#126270L, c#126279L, key#126280L, c#126291L]                                                                          Project [key#126270L, c#126279L, key#126280L, c#126291L]
2022-06-16T05:29:18.0666871Z [info]  +- Join Inner, (key#126270L = key#126280L)                                                                                        +- Join Inner, (key#126270L = key#126280L)
2022-06-16T05:29:18.0668143Z [info]     :- Filter isnotnull(key#126270L)                                                                                                  :- Filter isnotnull(key#126270L)
2022-06-16T05:29:18.0669257Z [info]     :  +- Project [c#126279L, key#126270L]                                                                                            :  +- Project [c#126279L, key#126270L]
2022-06-16T05:29:18.0670509Z [info]     :     +- Generate explode(c#126273), [1], false, [c#126279L]                                                                      :     +- Generate explode(c#126273), [1], false, [c#126279L]
2022-06-16T05:29:18.0671748Z [info] !   :        +- Filter ((size(c#126273, true) > 0) AND isnotnull(c#126273))                                                           :        +- Filter isnotnull(c#126273)
2022-06-16T05:29:18.0673102Z [info] !   :           +- Project [(id#126268L % 10) AS key#126270L, array(id#126268L, (id#126268L + 1), (id#126268L + 3)) AS c#126273]      :           +- Filter ((size(c#126273, true) > 0) AND isnotnull(c#126273))
2022-06-16T05:29:18.0674808Z [info] !   :              +- Filter (cast((id#126268L * 2) as int) > 1)                                                                      :              +- Project [(id#126268L % 10) AS key#126270L, array(id#126268L, (id#126268L + 1), (id#126268L + 3)) AS c#126273]
2022-06-16T05:29:18.0676143Z [info] !   :                 +- Range (0, 10, step=1, splits=Some(2))                                                                        :                 +- Filter (cast((id#126268L * 2) as int) > 1)
2022-06-16T05:29:18.0677658Z [info] !   +- Filter isnotnull(key#126280L)                                                                                                  :                    +- Range (0, 10, step=1, splits=Some(2))
2022-06-16T05:29:18.0678838Z [info] !      +- Project [c#126291L, key#126270L AS key#126280L]                                                                             +- Filter isnotnull(key#126280L)
2022-06-16T05:29:18.0680092Z [info] !         +- Generate explode(c#126273), [1], false, [c#126291L]                                                                         +- Project [c#126291L, key#126270L AS key#126280L]
2022-06-16T05:29:18.0681365Z [info] !            +- Filter ((size(c#126273, true) > 0) AND isnotnull(c#126273))                                                                 +- Generate explode(c#126273), [1], false, [c#126291L]
2022-06-16T05:29:18.0687264Z [info] !               +- Project [(id#126290L % 10) AS key#126270L, array(id#126290L, (id#126290L + 1), (id#126290L + 3)) AS c#126273]               +- Filter isnotnull(c#126273)
2022-06-16T05:29:18.0688660Z [info] !                  +- Range (0, 10, step=1, splits=Some(2))                                                                                       +- Filter ((size(c#126273, true) > 0) AND isnotnull(c#126273))
2022-06-16T05:29:18.0690609Z [info] !                                                                                                                                                    +- Project [(id#126290L % 10) AS key#126270L, array(id#126290L, (id#126290L + 1), (id#126290L + 3)) AS c#126273]
2022-06-16T05:29:18.0704834Z [info] !                                                                                                                                                       +- Range (0, 10, step=1, splits=Some(2))

Issue Links

• duplicates: SPARK-39520 ExpressionSetSuite test failure with Scala 2.13