Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6380

Resolution of equi-join key in post-join projection

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Auto Closed
    • None
    • None
    • SQL

    Description

      df1.join(df2, df1("key") === df2("key")).select("key")

      It would be great to just resolve key to df1("key") in the case of inner joins.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-2205 Unnecessary exchange operators in a join on multiple tables with the same join key.

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #6141 [Github] Pull Request #6141 (bryanjj)

          Activity

            People

              Unassigned Unassigned
              rxin Reynold Xin
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: