[CALCITE-6419] Invalid unparse for VARCHAR without precision in HiveSqlDialect And SparkSqlDialect

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.37.0
• Fix Version/s: 1.38.0
• Component/s: core
• Labels: pull-request-available
• Assignee: xiong duan
• Reporter: xiong duan
• Created: 24/May/24 08:12
• Updated: 17/Oct/24 22:41
• Resolved: 17/Oct/24 22:41

Description

When we execute SQL in Calcite:

select cast(product_id as varchar) from product;

Generage the HiveSQL\SparkSQL:

select cast(product_id as varchar) from product;

According to the https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-VarcharvarcharVarchar.

In Hive, the varchar must have a precision.So when unpare VARCHAR without precision, I will convert VARCHAR to String. VARCHAR with precison do nothing.

According to the https://spark.apache.org/docs/latest/sql-ref-datatypes.html.

In Spark, Same as Hive. But as note, It can only be used in table schema, not functions/operators. So I will convert VARCHAR with or without precision to String;

In SparkSQL, Varchar  with precision are not effective, but no error and just a warning:

spark-sql> select cast('value' as varchar(2));
24/05/24 16:04:39 WARN CharVarcharUtils: The Spark cast operator does not support char/varchar type and simply treats them as string type. Please use string type directly to avoid confusion. Otherwise, you can set spark.sql.legacy.charVarcharAsString to true, so that Spark treat them as string type as same as Spark 3.0 and earlier
value
Time taken: 2.797 seconds, Fetched 1 row(s)

Issue Links

• links to: GitHub Pull Request #3799