[SPARK-39518] Plan stability test is broken with Scala 2.13

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.4.0
• Component/s: SQL
• Assignee: Bruce Robbins
• Reporter: Hyukjin Kwon
• Created: 20/Jun/22 01:20
• Updated: 12/Dec/22 18:10
• Resolved: 20/Jun/22 23:07

Description

https://github.com/apache/spark/runs/6930230844?check_suite_focus=true

- check simplified (tpcds-v1.4/q89) *** FAILED *** (118 milliseconds)
[info]   Plans did not match:
[info]   last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q89/simplified.txt
[info]   last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q89/explain.txt
[info] 
[info]   TakeOrderedAndProject [sum_sales,avg_monthly_sales,s_store_name,i_category,i_class,i_brand,s_company_name,d_moy]
[info]     WholeStageCodegen (7)
[info]       Project [i_category,i_class,i_brand,s_store_name,s_company_name,d_moy,sum_sales,avg_monthly_sales]
[info]         Filter [avg_monthly_sales,sum_sales]
[info]           InputAdapter
[info]             Window [_w0,i_category,i_brand,s_store_name,s_company_name]
[info]               WholeStageCodegen (6)
[info]                 Sort [i_category,i_brand,s_store_name,s_company_name]
[info]                   InputAdapter
[info]                     Exchange [i_category,i_brand,s_store_name,s_company_name] #1
[info]                       WholeStageCodegen (5)
[info]                         HashAggregate [i_category,i_class,i_brand,s_store_name,s_company_name,d_moy,sum] [sum(UnscaledValue(ss_sales_price)),sum_sales,_w0,sum]
[info]                           InputAdapter
[info]                             Exchange [i_category,i_class,i_brand,s_store_name,s_company_name,d_moy] #2
[info]                               WholeStageCodegen (4)
[info]                                 HashAggregate [i_category,i_class,i_brand,s_store_name,s_company_name,d_moy,ss_sales_price] [sum,sum]
[info]                                   Project [i_brand,i_class,i_category,ss_sales_price,d_moy,s_store_name,s_company_name]
[info]                                     BroadcastHashJoin [ss_store_sk,s_store_sk]
[info]                                       Project [i_brand,i_class,i_category,ss_store_sk,ss_sales_price,d_moy]
[info]                                         BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
[info]                                           Project [i_brand,i_class,i_category,ss_store_sk,ss_sales_price,ss_sold_date_sk]
[info]                                             BroadcastHashJoin [i_item_sk,ss_item_sk]
[info]                                               Filter [i_category,i_class,i_item_sk]
[info]                                                 ColumnarToRow
[info]                                                   InputAdapter
[info]                                                     Scan parquet default.item [i_item_sk,i_brand,i_class,i_category]
[info]                                               InputAdapter
[info]                                                 BroadcastExchange #3
[info]                                                   WholeStageCodegen (1)
[info]                                                   InputAdapter
[info]                                                     Scan parquet default.item [i_item_sk,i_brand,i_class,i_category]
[info]                                               InputAdapter
[info]                                                 BroadcastExchange #3
[info]                                                   WholeStageCodegen (1)
[info]                                                     Filter [ss_item_sk,ss_store_sk]
[info]                                                       ColumnarToRow
[info]                                                         InputAdapter
[info]                                                           Scan parquet default.store_sales [ss_item_sk,ss_store_sk,ss_sales_price,ss_sold_date_sk]
[info]                                                             SubqueryBroadcast [d_date_sk] #1
[info]                                                               BroadcastExchange #4
[info]                                                                 WholeStageCodegen (1)
[info]                                                                   Project [d_date_sk,d_moy]
[info]                                                                     Filter [d_year,d_date_sk]
[info]                                                                       ColumnarToRow
[info]                                                                         InputAdapter
[info]                                                                           Scan parquet default.date_dim [d_date_sk,d_year,d_moy]
[info]                                           InputAdapter
[info]                                             ReusedExchange [d_date_sk,d_moy] #4
[info]                                       InputAdapter
[info]                                         BroadcastExchange #5
[info]                                           WholeStageCodegen (3)
[info]                                             Filter [s_store_sk]
[info]                                               ColumnarToRow
[info]                                                 InputAdapter
[info]                                                   Scan parquet default.store [s_store_sk,s_store_name,s_company_name] (PlanStabilitySuite.scala:158)

Issue Links

• duplicates: SPARK-39520 ExpressionSetSuite test failure with Scala 2.13