Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8947 Improve expression type coercion, casting & checking
  3. SPARK-9292

Analysis should check that join conditions' data types are booleans

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • SQL
    • None

    Description

      The following data frame query should fail analysis but instead fails at runtime:

          val df = Seq((1, 1)).toDF("a", "b")
          df.join(df, df.col("a"))
      

      This should fail with an AnalysisException because the column "A" is not a boolean and thus cannot be used as a join condition.

      This can be fixed by adding a new analysis rule which checks that the join condition has BooleanType.

      Attachments

        Activity

          People

            joshrosen Josh Rosen
            joshrosen Josh Rosen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: