Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21333

joinWith documents and analysis allow invalid join types

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.1, 2.2.0
    • Fix Version/s: 2.2.1, 2.3.0
    • Component/s: Documentation
    • Labels:
      None

      Description

      joinWith returns a Dataset<Tuple2>, but left_semi and left_anti (which only return things from one side of the join) are listed as acceptable join types in the documentation, and an error is not thrown until spark gets to the joining part, rather than in the analysis phase. These join types should be removed from the documentation, and this error should be caught earlier

        Attachments

          Activity

            People

            • Assignee:
              coreywoodfield Corey Woodfield
              Reporter:
              coreywoodfield Corey Woodfield
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified