Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23729

Glob resolution breaks remote naming of files/archives

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.1, 2.4.0
    • Component/s: Spark Submit
    • Labels:

      Description

      Given one uses spark-submit with either of the --archives or the --files parameters, in case the file name actually contains glob patterns, the rename part (...#nameAs) of the filename will eventually be ignored.

      Thinking over the resolution cases, if the resolution results in multiple files, it does not make sense to send all of them under the same remote name. So this should then result in an error.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              misutoth Mihaly Toth
              Reporter:
              misutoth Mihaly Toth

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment