[SPARK-15425] Disallow cartesian joins by default

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: SQL
• Labels: release_notes releasenotes
• Target Version/s: 2.0.0
• Assignee: Sameer Agarwal
• Reporter: Reynold Xin
• Created: 20/May/16 01:10
• Updated: 20/Jul/16 18:28
• Resolved: 23/May/16 06:32

Description

It is fairly easy for users to shoot themselves in the foot if they run cartesian joins. Often they might not even be aware of the join methods chosen. This happened to me a few times in the last few weeks.

It would be a good idea to disable cartesian joins by default, and require explicit enabling of it via "crossJoin" method or in SQL "cross join". This however might be too large of a scope for 2.0 given the timing. As a small and quick fix, we can just have a single config option (spark.sql.join.enableCartesian) that controls this behavior. In the future we can implement the fine-grained control.

Note that the error message should be friendly and say "Set spark.sql.join.enableCartesian to true to turn on cartesian joins."

Issue Links

• links to: [Github] Pull Request #13209 (sameeragarwal)
• links to: [Github] Pull Request #14057 (rxin)