Major Description
While trying to use a custom classpath for metastore jars (spark.sql.hive.metastore.jars pointing at some filesystem path), I ran into the following issue:
java.lang.ClassNotFoundException: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/MRVersion when creating Hive client using classpath
The issue here is that MRVersion is not packaged anywhere with Spark, and the code in IsolatedClientLoader only ever tries the parent class loader when loading hadoop classes in this configuration. So even though I had the class in the list of files in spark.sql.hive.metastore.jars, Spark never tries to load it.
