[SPARK-23567] spark.redaction.regex should not include user by default, docs not updated

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.2.1
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Thomas Graves
• Created: 02/Mar/18 15:49
• Updated: 04/Apr/18 19:15
• Resolved: 04/Apr/18 19:15

Description

SPARK-22479 changed to redact the user name by default.  I would argue username isn't something that should be redacted by default and its very useful for debugging and other things. If people are running super secure and want to turn it on they can but I don't see the user name as a default security setting.  There are also other ways on the UI to see the user name, for instance on yarn you can go to the Environment page and looking at the resources and see the username in the paths.

Also the Jira did not update the default setting in the docs, so the docs are out of date:

http://spark.apache.org/docs/2.2.1/configuration.html

Issue Links

• is broken by: SPARK-22479 SaveIntoDataSourceCommand logs jdbc credentials
• is duplicated by: SPARK-23850 We should not redact username|user|url from UI by default