[FLINK-29152] Describe statement resutls is different from the Hive

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0, 1.17.0
• Component/s: Connectors / Hive
• Labels: pull-request-available
• Assignee: luoyuxia
• Reporter: Shengkai Fang
• Created: 31/Aug/22 06:50
• Updated: 21/Sep/22 15:10
• Resolved: 21/Sep/22 15:10

Description

In hive, the results schema is 

+-----------+------------+----------+
| col_name  | data_type  | comment  |
+-----------+------------+----------+
| a         | int        |          |
| b         | string     |          |
+-----------+------------+----------+

but our implementation is

0: jdbc:hive2://localhost:10000/default> describe sink;
+-------+-------+-------+-------+---------+------------+
| name  | type  | null  |  key  | extras  | watermark  |
+-------+-------+-------+-------+---------+------------+
| a     | INT   | true  | NULL  | NULL    | NULL       |
+-------+-------+-------+-------+---------+------------+

BTW, it's better we can support DESCRIBE FORMATTED like hive does.

+-------------------------------+----------------------------------------------------+-----------------------+
|           col_name            |                     data_type                      |        comment        |
+-------------------------------+----------------------------------------------------+-----------------------+
| # col_name                    | data_type                                          | comment               |
|                               | NULL                                               | NULL                  |
| a                             | int                                                |                       |
| b                             | string                                             |                       |
|                               | NULL                                               | NULL                  |
| # Detailed Table Information  | NULL                                               | NULL                  |
| Database:                     | default                                            | NULL                  |
| Owner:                        | null                                               | NULL                  |
| CreateTime:                   | Tue Aug 30 06:54:00 UTC 2022                       | NULL                  |
| LastAccessTime:               | UNKNOWN                                            | NULL                  |
| Retention:                    | 0                                                  | NULL                  |
| Location:                     | hdfs://namenode:8020/user/hive/warehouse/sink      | NULL                  |
| Table Type:                   | MANAGED_TABLE                                      | NULL                  |
| Table Parameters:             | NULL                                               | NULL                  |
|                               | comment                                            |                       |
|                               | numFiles                                           | 0                     |
|                               | totalSize                                          | 0                     |
|                               | transient_lastDdlTime                              | 1661842440            |
|                               | NULL                                               | NULL                  |
| # Storage Information         | NULL                                               | NULL                  |
| SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                  |
| InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                  |
| OutputFormat:                 | org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat | NULL                  |
| Compressed:                   | No                                                 | NULL                  |
| Num Buckets:                  | -1                                                 | NULL                  |
| Bucket Columns:               | []                                                 | NULL                  |
| Sort Columns:                 | []                                                 | NULL                  |
| Storage Desc Params:          | NULL                                               | NULL                  |
|                               | serialization.format                               | 1                     |
+-------------------------------+----------------------------------------------------+-----------------------+

Issue Links

• relates to: FLINK-28952 Release Testing: Veify Hive dialect
• links to: GitHub Pull Request #20789
• links to: GitHub Pull Request #20795