[SPARK-20576] Support generic hint function in Dataset/DataFrame

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 03/May/17 06:02
• Updated: 05/May/17 06:14
• Resolved: 05/May/17 06:14

Description

We allow users to specify hints (currently only "broadcast" is supported) in SQL and DataFrame. However, while SQL has a standard hint format (/*+ ... */), DataFrame doesn't have one and sometimes users are confused that they can't find how to apply a broadcast hint. This ticket adds a generic hint function on DataFrame that allows using the same hint on DataFrames as well as SQL.

As an example, the following will apply a broadcast hint on a DataFrame using the new hint function:

df1.join(df2.hint("broadcast"))

Issue Links

• links to: [Github] Pull Request #17839 (rxin)