Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23079

Fix query constraints propagation with aliases

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.1
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Previously, PR #19201 fix the problem of non-converging constraints.

      After that PR #19149 improve the loop and constraints is inferred only once.

      So the problem of non-converging constraints is gone.

      Also, in current code, the case below will fail.

      ```

      spark.range(5).write.saveAsTable("t")
      val t = spark.read.table("t")
      val left = t.withColumn("xid", $"id" + lit(1)).as("x")
      val right = t.withColumnRenamed("id", "xid").as("y")
      val df = left.join(right, "xid").filter("id = 3").toDF()
      checkAnswer(df, Row(4, 3))

      ```

      Because of `aliasMap` replace all the aliased child. See the test case in PR for details.

       

       

        Attachments

          Activity

            People

            • Assignee:
              Gengliang.Wang Gengliang Wang
              Reporter:
              Gengliang.Wang Gengliang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: