Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13105

Spark 1.6 and earlier should reject NATURAL JOIN queries instead of returning wrong answers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 1.4.1, 1.5.2, 1.6.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      In Spark 1.6 and earlier, Spark SQL does not support NATURAL JOIN queries. However, its SQL parser does not consider NATURAL to be a reserved word, which causes natural joins to be parsed as regular joins where the left table has been aliased. For instance,

      SELECT * FROM foo NATURAL JOIN bar
      

      gets interpreted as "foo JOIN bar" where "foo" is aliased to "natural".

      Rather than doing this, which leads to confusing / wrong results for users who expect NATURAL JOIN behavior, Spark should immediately reject these queries at analysis time and should provide an informative error message.

      We're going to add natural join support in Spark 2.0, but for earlier versions we should add a bugfix to throw errors.

        Attachments

          Activity

            People

            • Assignee:
              joshrosen Josh Rosen
              Reporter:
              joshrosen Josh Rosen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: