Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30263

Don't log values of ignored non-Spark properties

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.4, 3.0.0
    • 2.4.5, 3.0.0
    • Spark Core
    • None

    Description

      Comment per Aaron Steers:

      Is it expected that this error would print aws security keys to log files? Seems like a serious security concern.

      Warning: Ignoring non-spark config property: fs.s3a.access.key={full-access-key}
      Warning: Ignoring non-spark config property: fs.s3a.secret.key={full-secret-key}
      

      Could we not accomplish the same thing by printing the name of the key without the key's value?

      I think we can also redact these, but, also no big reason to log the value of ignored properties here anyway.

      Attachments

        Issue Links

          Activity

            People

              srowen Sean R. Owen
              srowen Sean R. Owen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: