[SPARK-11803] Dataset self join returns incorrect result

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: SQL
• Target Version/s: 1.6.0
• Assignee: Wenchen Fan
• Reporter: Reynold Xin
• Created: 18/Nov/15 02:57
• Updated: 24/Mar/17 05:17
• Resolved: 18/Nov/15 18:16

Description

See the test case in https://github.com/apache/spark/pull/9789

  ignore("self join") {
    val ds = Seq("1", "2").toDS().as("a")
    val joined = ds.joinWith(ds, lit(true))
    checkAnswer(joined, ("1", "1"), ("1", "2"), ("2", "1"), ("2", "2"))
  }

Issue Links

• relates to: SPARK-20073 Unexpected Cartesian product when using eqNullSafe in join with a derived table
• links to: [Github] Pull Request #9806 (cloud-fan)