Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30263

Don't log values of ignored non-Spark properties

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.4, 3.0.0
    • Fix Version/s: 2.4.5, 3.0.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Comment per Aaron Steers:

      Is it expected that this error would print aws security keys to log files? Seems like a serious security concern.

      Warning: Ignoring non-spark config property: fs.s3a.access.key={full-access-key}
      Warning: Ignoring non-spark config property: fs.s3a.secret.key={full-secret-key}
      

      Could we not accomplish the same thing by printing the name of the key without the key's value?

      I think we can also redact these, but, also no big reason to log the value of ignored properties here anyway.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srowen Sean R. Owen
                Reporter:
                srowen Sean R. Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: