Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23729

Glob resolution breaks remote naming of files/archives

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.1, 2.4.0
    • Component/s: Spark Submit
    • Labels:

      Description

      Given one uses spark-submit with either of the --archives or the --files parameters, in case the file name actually contains glob patterns, the rename part (...#nameAs) of the filename will eventually be ignored.

      Thinking over the resolution cases, if the resolution results in multiple files, it does not make sense to send all of them under the same remote name. So this should then result in an error.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                misutoth Mihaly Toth
                Reporter:
                misutoth Mihaly Toth
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: