Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16501

spark.mesos.secret exposed on UI and command line

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2
    • Fix Version/s: 2.4.0
    • Component/s: Spark Submit, Web UI
    • Labels:

      Description

      There are two related problems with spark.mesos.secret:

      1) The web UI shows its value in the "environment" tab
      2) Passing it as a command-line option to spark-submit (or creating a SparkContext from python, with the effect of launching spark-submit) exposes it to "ps"

      I'll be happy to submit a patch but I could use some advice first.

      The first problem is easy enough, just don't show that value in the UI

      For the second problem, I'm not sure what the best solution is. A "spark.mesos.secret-file" parameter would let the user store the secret in a non-world-readable file. Alternatively, the mesos secret could be obtained from the environment, which other users don't have access to. Either solution would work in client mode, but I don't know if they're workable in cluster mode.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rvesse Rob Vesse
                Reporter:
                edaniel Eric Daniel
              • Votes:
                2 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: