[SPARK-21697] NPE & ExceptionInInitializerError trying to load UDF from HDFS

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.1.1
• Component/s: SQL

Environment

Spark Client mode, Hadoop 2.6.0

• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 10/Aug/17 19:45
• Updated: 21/Apr/22 14:42
• Resolved: 01/Nov/19 19:27

Description

Reported on the PR for SPARK-12868: trying to load a UDF of HDFS is triggering an ExceptionInInitializerError, caused by an NPE which should only happen if the commons-logging LOG log is null.

Hypothesis: the commons logging scan for commons-logging.properties is happening in the classpath with the HDFS JAR; this is triggering a D/L of the JAR, which needs to force in commons-logging, and, as that's not inited yet, NPEs

Issue Links

• duplicates: SPARK-12868 ADD JAR via sparkSQL JDBC will fail when using a HDFS URL
• links to: [Github] Pull Request #36301 (cxzl25)
• links to: [Github] Pull Request #36301 (cxzl25)