Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-778

Livy does not respect spark-defaults.conf configuration

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • 0.9.0
    • Server
    • None

    Description

      Livy currently does not respect the spark.yarn.dist.archives property even when it's specified in
      spark-defaults.conf, due to this code block:
      https://github.com/apache/incubator-livy/blob/b8251eb9b3d63c77c61a5950dee5958b654d9633/rsc/src/main/java/org/apache/livy/rsc/ContextLauncher.java#L295-L301
       
      There should be a way to configure the Livy server with additional spark
      configuration for the property spark.yarn.dist.archives

      Attachments

        Activity

          People

            Unassigned Unassigned
            jpugliesi John Pugliesi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: