[SPARK-35576] Redact the sensitive info in the result of Set command

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.3, 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.8, 3.0.2, 3.1.2, 3.2.0
• Fix Version/s: 3.2.0, 3.1.3
• Component/s: Security, SQL
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 31/May/21 10:51
• Updated: 01/Jun/21 07:33
• Resolved: 31/May/21 22:00

Description

Currently, the results of following SQL queries are not redacted:
```
SET [KEY];
SET;
```
For example:

scala> spark.sql("set javax.jdo.option.ConnectionPassword=123456").show()
+--------------------+------+
|                 key| value|
+--------------------+------+
|javax.jdo.option....|123456|
+--------------------+------+


scala> spark.sql("set javax.jdo.option.ConnectionPassword").show()
+--------------------+------+
|                 key| value|
+--------------------+------+
|javax.jdo.option....|123456|
+--------------------+------+


scala> spark.sql("set").show()
+--------------------+--------------------+
|                 key|               value|
+--------------------+--------------------+
|javax.jdo.option....|              123456|

We should hide the sensitive information and redact the query output.

Issue Links

• links to: [Github] Pull Request #32712 (gengliangwang)
• links to: [Github] Pull Request #32712 (gengliangwang)
• links to: [Github] Pull Request #32720 (gengliangwang)
• links to: [Github] Pull Request #32720 (gengliangwang)