Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13105

Spark 1.6 and earlier should reject NATURAL JOIN queries instead of returning wrong answers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 1.4.1, 1.5.2, 1.6.0
    • None
    • SQL
    • None

    Description

      In Spark 1.6 and earlier, Spark SQL does not support NATURAL JOIN queries. However, its SQL parser does not consider NATURAL to be a reserved word, which causes natural joins to be parsed as regular joins where the left table has been aliased. For instance,

      SELECT * FROM foo NATURAL JOIN bar
      

      gets interpreted as "foo JOIN bar" where "foo" is aliased to "natural".

      Rather than doing this, which leads to confusing / wrong results for users who expect NATURAL JOIN behavior, Spark should immediately reject these queries at analysis time and should provide an informative error message.

      We're going to add natural join support in Spark 2.0, but for earlier versions we should add a bugfix to throw errors.

      Attachments

        Activity

          People

            joshrosen Josh Rosen
            joshrosen Josh Rosen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: