Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19042

Remove query string from jar url for executor

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.2.0
    • None
    • None

    Description

      spark.jars support jar url with http protocal. However, if the url contains any query strings, the "localName = name.split("/").last" won't get the expected jar, then "val url = new File(SparkFiles.getRootDirectory(), localName).toURI.toURL" will get invalid url. The bug fix is the same as SPARK-17855

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hustfxj John Fang
            hustfxj John Fang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment