[SPARK-29247] HiveClientImpl may be log sensitive information

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: SQL
• Assignee: angerszhu Assign to me
• Reporter: Yuming Wang
• Created: 25/Sep/19 15:58
• Updated: 29/Sep/19 21:33
• Resolved: 29/Sep/19 21:33

Description

HiveClientImpl may be log sensitive information. e.g. url, secret and token:

      logDebug(
        s"""
           |Applying Hadoop/Hive/Spark and extra properties to Hive Conf:
           |$k=${if (k.toLowerCase(Locale.ROOT).contains("password")) "xxx" else v}
         """.stripMargin)

We need to redact it. Please refer to SQLConf.get.redactOptions.

https://github.com/apache/spark/blob/c8628354b7d2e6116b2a6eb3bdb2fc957c91fd03/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala#L181-L185

Issue Links

• links to: GitHub Pull Request #25954