[SPARK-42946] Sensitive data could still be exposed by variable substitution

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.2, 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Kent Yao
• Reporter: Kent Yao
• Created: 28/Mar/23 08:48
• Updated: 29/Mar/23 04:38
• Resolved: 29/Mar/23 04:38

Description

Case 1 by SET syntax's key part

 

Time taken: 0.017 seconds, Fetched 1 row(s)
spark-sql> set ${spark.ssl.keyPassword}
         > ;
abc    <undefined> 

Case 2 by SELECT as String lit

 

spark-sql> set spark.ssl.keyPassword;
spark.ssl.keyPassword    *********(redacted)
Time taken: 0.009 seconds, Fetched 1 row(s)
spark-sql> select '${spark.ssl.keyPassword}'
         > ;
abc