[SPARK-27244] Redact Passwords While Using Option logConf=true

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.0, 2.3.2, 2.3.3, 2.4.0
• Fix Version/s: 2.3.4, 2.4.2, 3.0.0
• Component/s: Spark Core
• Assignee: Ninad
• Reporter: Ninad
• Created: 22/Mar/19 10:49
• Updated: 31/Mar/19 07:08
• Resolved: 29/Mar/19 19:27

Description

When logConf is set to true and any sensitive information like passwords is set using spark-submit command then the passwords are logged in clear text from conf. Ideally, these passwords should be redacted as it does on UI and then printed to log for debugging purpose.

Issue Links

• links to: GitHub Pull Request #24196
• links to: GitHub Pull Request #24257