[SPARK-49179] Fix v2 multi bucketed inner joins throw AssertionError

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0, 3.3.4, 3.5.2, 3.4.3
• Fix Version/s: 4.0.0, 3.4.4, 3.5.3
• Component/s: SQL
• Labels: pull-request-available
• Assignee: XiDuo You
• Reporter: XiDuo You
• Created: 09/Aug/24 07:52
• Updated: 13/Aug/24 09:01
• Resolved: 13/Aug/24 05:10

Description

[info]   Cause: java.lang.AssertionError: assertion failed
[info]   at scala.Predef$.assert(Predef.scala:264)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.createKeyGroupedShuffleSpec(EnsureRequirements.scala:642)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.$anonfun$checkKeyGroupCompatible$1(EnsureRequirements.scala:385)
[info]   at scala.collection.immutable.List.map(List.scala:247)
[info]   at scala.collection.immutable.List.map(List.scala:79)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.checkKeyGroupCompatible(EnsureRequirements.scala:382)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.checkKeyGroupCompatible(EnsureRequirements.scala:364)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.org$apache$spark$sql$execution$exchange$EnsureRequirements$$ensureDistributionAndOrdering(EnsureRequirements.scala:166)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements$$anonfun$1.applyOrElse(EnsureRequirements.scala:714)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements$$anonfun$1.applyOrElse(EnsureRequirements.scala:689)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformUpWithPruning$4(TreeNode.scala:528)
[info]   at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:84)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.transformUpWithPruning(TreeNode.scala:528)
[info]   at org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:497)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.apply(EnsureRequirements.scala:689)
[info]   at org.apache.spark.sql.execution.exchange.EnsureRequirements.apply(EnsureRequirements.scala:51)
[info]   at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec$.$anonfun$applyPhysicalRules$2(AdaptiveSparkPlanExec.scala:882)
[in

Issue Links

• links to: GitHub Pull Request #47683
• links to: GitHub Pull Request #47735
• links to: GitHub Pull Request #47736