[HUDI-4799] improve analyzer exception tip when can not resolve expression

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.2
• Component/s: spark-sql
• Labels: pull-request-available
• Assignee: KnightChess
• Reporter: KnightChess
• Created: 07/Sep/22 14:21
• Updated: 31/Oct/22 07:41
• Resolved: 31/Oct/22 07:41

Description

sql:
merge into hudi_mor_pk_cbfield_tbl3 as target 
using (select id, dt from delete_sync_test where concat_ws('1', '2') and id = 1) as source 
on target.id = source.id 
when matched then update set target.ts = source.dt

the source table is unresolved

then Exception desc is doubtful

Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot resolve 'target.id,'source.id in (target.id = source.id), the input columns is: ['id, 'dt, _hoodie_commit_time#0, _hoodie_commit_seqno#1, _hoodie_record_key#2, _hoodie_partition_path#3, _hoodie_file_name#4, id#5L, ts#6L, name#7]
	at org.apache.spark.sql.hudi.analysis.HoodieResolveReferences.org$apache$spark$sql$hudi$analysis$HoodieResolveReferences$$resolveExpressionFrom(HoodieAnalysis.scala:568)
	at org.apache.spark.sql.hudi.analysis.HoodieResolveReferences$$anonfun$apply$1.applyOrElse(HoodieAnalysis.scala:351)
	at org.apache.spark.sql.hudi.analysis.HoodieResolveReferences$$anonfun$apply$1.applyOrElse(HoodieAnalysis.scala:265)

after imporve:
it is easy find error

*Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve '(concat_ws('1', '2') AND (spark_catalog.hudi_test.delete_sync_test.id = 1))' due to data type mismatch: differing types in '(concat_ws('1', '2') AND (spark_catalog.hudi_test.delete_sync_test.id = 1))' (string and boolean).; line 2 pos 49;
'SubqueryAlias source
+- 'Project ['id, 'dt]
   +- 'Filter (concat_ws(1, 2) AND (id#13 = 1))
      +- SubqueryAlias spark_catalog.hudi_test.delete_sync_test
         +- Relation hudi_test.delete_sync_test[_hoodie_commit_time#8,_hoodie_commit_seqno#9,_hoodie_record_key#10,_hoodie_partition_path#11,_hoodie_file_name#12,id#13,age#14,name#15,dt#16] parquet

	at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
	at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$$nestedInanonfun$checkAnalysis$1$2.applyOrElse(CheckAnalysis.scala:190)

 

 

Issue Links

• links to: GitHub Pull Request #6625