Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.2.0
-
None
Description
I am calling
sparkContext.addJar
method with path to a local jar I want to add. Example:
/home/me/.coursier/cache/v1/https/artifactory.com%3A443/path/to.jar
. As a result I get an exception saying
Failed to add /home/me/.coursier/cache/v1/https/artifactory.com%3A443/path/to.jar to Spark environment. Stacktrace: java.io.FileNotFoundException: Jar /home/me/.coursier/cache/v1/https/artifactory.com:443/path/to.jar not found
Important part to notice here is that colon character is url encoded in path I want to use but exception is complaining about path in decoded form. This is caused by this line of code from implementation (see here):
case null | "file" => addJarFile(new File(uri.getPath))
It uses getPath method of java.net.URI which url decodes the path. I believe method getRawPath should be used here which keeps path string in original form.
I tend to see this as a bug since I want to use my dependencies resolved from artifactory with port directly. Is there some specific reason for this or can we fix this?
Thanks