[SPARK-23850] We should not redact username|user|url from UI by default

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.1
• Fix Version/s: 2.2.2, 2.3.1, 2.4.0
• Component/s: Web UI
• Assignee: Marcelo Masiero Vanzin
• Reporter: Thomas Graves
• Created: 02/Apr/18 18:47
• Updated: 09/Aug/19 07:23
• Resolved: 18/May/18 23:25

Description

SPARK-22479 was filed to not print the log jdbc credentials, but in there they also added  the username and url to be redacted.  I'm not sure why these were added and to me by default these do not have security concerns.  It makes it more usable by default to be able to see these things.  Users with high security concerns can simply add them in their configs.

Also on yarn just redacting url doesn't secure anything because if you go to the environment ui page you see all sorts of paths and really its just confusing that some of its redacted and other parts aren't.  If this was specifically for jdbc I think it needs to be just applied there and not broadly.

If we remove these we need to test what the jdbc driver is going to log from SPARK-22479.

Issue Links

• duplicates: SPARK-23567 spark.redaction.regex should not include user by default, docs not updated
• is broken by: SPARK-22479 SaveIntoDataSourceCommand logs jdbc credentials
• relates to: SPARK-28675 Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions
• links to: [Github] Pull Request #21158 (vanzin)
• links to: [Github] Pull Request #21365 (vanzin)