[SPARK-36398] Redact sensitive information in Spark Thrift Server log

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.2
• Fix Version/s: 3.2.0, 3.1.3
• Component/s: Security, SQL
• Assignee: Kousuke Saruta
• Reporter: Denis Krivenko
• Created: 03/Aug/21 22:04
• Updated: 25/Aug/21 12:33
• Resolved: 25/Aug/21 12:33

Description

Spark Thrift Server logs query without sensitive information redaction in org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.scala

  override def runInternal(): Unit = {
    setState(OperationState.PENDING)
    logInfo(s"Submitting query '$statement' with $statementId")

Logs

21/08/03 20:49:46 INFO SparkExecuteStatementOperation: Submitting query 'CREATE OR REPLACE TEMPORARY VIEW test_view
    USING org.apache.spark.sql.jdbc
    OPTIONS (
        url="jdbc:mysql://example.com:3306",
        driver="com.mysql.jdbc.Driver",
        dbtable="example.test",
        user="my_username",
        password="my_password"
    )' with 37e5d2cb-aa96-407e-b589-7cb212324100
21/08/03 20:49:46 INFO SparkExecuteStatementOperation: Running query with 37e5d2cb-aa96-407e-b589-7cb212324100

Issue Links

• links to: [Github] Pull Request #33743 (sarutak)
• links to: [Github] Pull Request #33832 (sarutak)