Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21012

Support glob path for resources adding to Spark

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • Spark Submit
    • None

    Description

      Current "--jars (spark.jars)", "--files (spark.files)", "--py-files (spark.submit.pyFiles)" and "--archives (spark.yarn.dist.archives)" only support non-glob path. This is OK for most of the cases, but when user requires to add more jars, files into Spark, it is too verbose to list one by one. So here propose to support glob path for adding resources.

      Also improving the code of downloading resources.

      Attachments

        Issue Links

          Activity

            People

              jerryshao Saisai Shao
              jerryshao Saisai Shao
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: