[SPARK-26602] Subsequent queries are failing after querying the UDF which is loaded with wrong hdfs path

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.4.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Haripriya
• Created: 11/Jan/19 11:26
• Updated: 12/Dec/22 18:11
• Resolved: 06/Mar/19 15:01

Description

In sql,

1.Query the existing  udf(say myFunc1)

2. create and select the udf registered with incorrect path (say myFunc2)

3.Now again query the existing udf  in the same session - Wil throw exception stating that couldn't read resource of myFunc2's path

4.Even  the basic operations like insert and select will fail giving the same error

Result: 

java.lang.RuntimeException: Failed to read external resource hdfs:///tmp/hari_notexists1/two_udfs.jar
at org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288)
at org.apache.hadoop.hive.ql.session.SessionState.resolveAndDownload(SessionState.java:1242)
at org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1163)
at org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1149)
at org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:67)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:737)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:706)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:275)
at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:213)
at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:212)
at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:258)
at org.apache.spark.sql.hive.client.HiveClientImpl.runHive(HiveClientImpl.scala:706)
at org.apache.spark.sql.hive.client.HiveClientImpl.runSqlHive(HiveClientImpl.scala:696)
at org.apache.spark.sql.hive.client.HiveClientImpl.addJar(HiveClientImpl.scala:841)
at org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:112)

Issue Links

• duplicates: SPARK-26560 Repeating select on udf function throws analysis exception - function not registered
• links to: GitHub Pull Request #23972