[SPARK-10972] UDFs in SQL joins

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.5.1
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Michael Malak
• Created: 07/Oct/15 15:04
• Updated: 21/May/19 04:34
• Resolved: 21/May/19 04:34

Description

Currently expressions used to .join() in DataFrames are limited to column names plus the operators exposed in org.apache.spark.sql.Column.

It would be nice to be able to .join() based on a UDF, such as, say, euclideanDistance(col1, col2) < 0.1.