[SPARK-22314] Accessing Hive UDFs defined without 'USING JAR' from Spark

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.2.0
• Component/s: Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Matyas Orhidi
• Created: 19/Oct/17 17:38
• Updated: 21/May/19 04:14
• Resolved: 21/May/19 04:14

Description

When defining UDF functions in Hive it is possible to load the UDF jar(s) from a shared location e.g. from hive.reloadable.aux.jars.path, and then use the CREATE FUNCTION statement:

CREATE FUNCTION <your_function_name> AS '<fully_qualified_class_name>';

These UDFs are not working from Spark unless you use the

CREATE FUNCTION <your_function_name> AS '<fully_qualified_class_name>' USING JAR 'hdfs:///<path/to/jar/in/hdfs>';
command to create the Hive UDF function.