Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21333

joinWith documents and analysis allow invalid join types

Rank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.1.1, 2.2.0
    • 2.2.1, 2.3.0
    • Documentation
    • None

    Description

      joinWith returns a Dataset<Tuple2>, but left_semi and left_anti (which only return things from one side of the join) are listed as acceptable join types in the documentation, and an error is not thrown until spark gets to the joining part, rather than in the analysis phase. These join types should be removed from the documentation, and this error should be caught earlier

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            coreywoodfield Corey Woodfield
            coreywoodfield Corey Woodfield
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                Slack

                  Issue deployment