[FLINK-32249] A Java string should be used instead of a Calcite NlsString to construct the column comment of CatalogTable

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.17.1
• Fix Version/s: 1.17.2
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: lincoln lee
• Reporter: lincoln lee
• Created: 04/Jun/23 15:38
• Updated: 06/Jun/23 00:53
• Resolved: 06/Jun/23 00:53

Description

when Flink interacts with CatalogTable, it directly passes the Calcite's NlsString comment as a string to the comment attribute of the schema and column. In theory, a Java string should be passed here, otherwise the CatalogTable implementers may encounter special character encoding issues, e.g., an issue in apache paimon: https://github.com/apache/incubator-paimon/issues/1262

also tested in sql-client:

Flink SQL> CREATE TABLE s1 (
>     order_id    STRING comment '测试中文',
>     price       DECIMAL(32,2) comment _utf8'测试_utf8中文',
>     currency    STRING,
>     order_time  TIMESTAMP(3)
> ) comment '测试中文table comment' WITH ('connector'='dategen');
[INFO] Execute statement succeed.

Flink SQL> show tables;
+------------+
| table name |
+------------+
|         s1 |
+------------+
1 row in set

Flink SQL> desc s1;
+------------+----------------+------+-----+--------+-----------+-------------------------+
|       name |           type | null | key | extras | watermark |                 comment |
+------------+----------------+------+-----+--------+-----------+-------------------------+
|   order_id |         STRING | TRUE |     |        |           | u&'\6d4b\8bd5\4e2d\6587 |
|      price | DECIMAL(32, 2) | TRUE |     |        |           |     _UTF8'测试_utf8中文 |
|   currency |         STRING | TRUE |     |        |           |                         |
| order_time |   TIMESTAMP(3) | TRUE |     |        |           |                         |
+------------+----------------+------+-----+--------+-----------+-------------------------+
4 rows in set

Flink SQL> show create table s1;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                                                                   result |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CREATE TABLE `default_catalog`.`default_database`.`s1` (
  `order_id` VARCHAR(2147483647),
  `price` DECIMAL(32, 2),
  `currency` VARCHAR(2147483647),
  `order_time` TIMESTAMP(3)
) COMMENT '测试中文table comment'
WITH (
  'connector' = 'dategen'
)
 |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set

Issue Links

• links to: GitHub Pull Request #22715