[SPARK-34128] Suppress excessive logging of TTransportExceptions in Spark ThriftServer

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.1, 3.1.0
• Fix Version/s: 3.1.2, 3.2.0
• Component/s: SQL
• Assignee: Kent Yao 2
• Reporter: Kent Yao 2
• Created: 15/Jan/21 12:46
• Updated: 20/Mar/21 04:18
• Resolved: 20/Mar/21 04:18

Description

Since Spark 3.0， the `libthrift` has been bumped up from 0.9.3 to 0.12.0.

Due to THRIFT-4805, The SparkThrift Server will print annoying TExceptions. For example, the current thrift server module test in Github action workflow outputs more than 200MB of data for this error only, while the total size of the test log only about 1GB.

 

I checked the latest `hive-service-rpc` module in the maven center,  https://mvnrepository.com/artifact/org.apache.hive/hive-service-rpc/3.1.2.  It still uses the 0.9.3 version. 

 

Due to THRIFT-5274 , It looks like we need to wait for thrift 0.14.0 to release or downgrade to 0.9.3 to fix this issue if any of them is appropriate

 

Issue Links

• is related to: THRIFT-4805 Suppress excessive logging of SASL TTransportExceptions in case of END_OF_FILE
• is related to: THRIFT-5274 Thrift 0.13.0 does not work with JDK8
• links to: [Github] Pull Request #31895 (yaooqinn)
• links to: [Github] Pull Request #31895 (yaooqinn)