[HIVE-26667] Incompatible expression deserialization against latest HMS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-1
• Component/s: Standalone Metastore
• Labels: pull-request-available
• Assignee: Zhihua Deng
• Reporter: Zhihua Deng
• Created: 25/Oct/22 02:33
• Updated: 10/Nov/22 21:38
• Resolved: 10/Nov/22 21:38

Description

When an old Metastore client requests listPartitionsByExpr against the lastest HMS, an exception would be thrown:

 

MetaException(message:Unable to find class: 
)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result.read(ThriftHiveMetastore.java)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_by_expr(ThriftHiveMetastore.java:3273)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_by_expr(ThriftHiveMetastore.java:3260)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByExpr(HiveMetaStoreClient.java:1488)

This was caused by a gap between old client and server on (de)serializing the expression. Old client doesn't stream the expression’s class type into bytes, while HMS reads the class type from serialized bytes first, which makes the trouble. Other APIs that need to (de)serialize expression may be suffered as well.

 

Issue Links

• links to: GitHub Pull Request #3700