Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33778

Allow typesafe join for LeftSemi and LeftAnti

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.0.1
    • None
    • SQL
    • None

    Description

      With SPARK-21333 change, LeftSemi and LeftAnti no longer has a typesafe join API. It makes sense to not support LeftSemi and LeftAnti as part of joinWith as it returns tuples which includes values from both the datasets which is not possible in the above joins. Neverthless, it would be nice to have a separate join API or in the existing API to support LeftSemi and LeftAnti which returns Dataset.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-21333 joinWith documents and analysis allow invalid join types

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #30775 [Github] Pull Request #30775 (AngersZhuuuu)

          Activity

            People

              Unassigned Unassigned
              vsowrirajan Venkata krishnan Sowrirajan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: