[HIVE-11837] comments do not support unicode characters well.

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.13.1, 1.1.0
• Component/s: Hive

Environment

Hadoop 2.7
Hive 0.13.1 / Hive 1.1.0
RHEL 6.4 / SLES 11.3

• Assignee: Rudd Chen
• Reporter: Rudd Chen
• Due: 16/Sep/15
• Created: 16/Sep/15 06:27
• Updated: 29/Mar/17 13:22
• Resolved: 29/Mar/17 13:22

Description

the terminal encoding is set to UTF-8, It can display Chinese characters.

then I create a table with a comment in Chinese, both "show create table" and "desc formatted table" can not display the Chinese characters in the table comments, meanwhile it can display Chinese characters in column comment.. See below:

0: jdbc:hive2://ha-cluster/default> create table tt(id int comment '列中文测试') comment '表中文测试';
No rows affected (0.152 seconds)
0: jdbc:hive2://ha-cluster/default>
0: jdbc:hive2://ha-cluster/default>
0: jdbc:hive2://ha-cluster/default> desc formatted tt;
-------------------------------------------------------------------------------------------------------------------

col_name

data_type

comment

-------------------------------------------------------------------------------------------------------------------

col_name	data_type	comment
	NULL	NULL
id	int	列中文测试
	NULL	NULL
Detailed Table Information	NULL	NULL
Database:	default	NULL
Owner:	admin	NULL
CreateTime:	Wed Sep 16 11:13:34 CST 2015	NULL
LastAccessTime:	UNKNOWN	NULL
Protect Mode:	None	NULL
Retention:	0	NULL
Location:	hdfs://hacluster/user/hive/warehouse/tt	NULL
Table Type:	MANAGED_TABLE	NULL
Table Parameters:	NULL	NULL
	comment	\u8868\u4E2D\u6587\u6D4B\u8BD5
	transient_lastDdlTime	1442373214
	NULL	NULL
Storage Information	NULL	NULL
SerDe Library:	org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe	NULL
InputFormat:	org.apache.hadoop.hive.ql.io.RCFileInputFormat	NULL
OutputFormat:	org.apache.hadoop.hive.ql.io.RCFileOutputFormat	NULL
Compressed:	No	NULL
Num Buckets:	-1	NULL
Bucket Columns:	[]	NULL
Sort Columns:	[]	NULL
Storage Desc Params:	NULL	NULL
	serialization.format	1

-------------------------------------------------------------------------------------------------------------------
27 rows selected (0.172 seconds)
0: jdbc:hive2://ha-cluster/default> show create table tt;
------------------------------------------------------------

createtab_stmt

------------------------------------------------------------

CREATE TABLE `tt`(

`id` int COMMENT '-�K�')

COMMENT 'h-�K�'

ROW FORMAT SERDE

'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'

STORED AS INPUTFORMAT

'org.apache.hadoop.hive.ql.io.RCFileInputFormat'

OUTPUTFORMAT

'org.apache.hadoop.hive.ql.io.RCFileOutputFormat'

LOCATION

'hdfs://hacluster/user/hive/warehouse/tt'

TBLPROPERTIES (

'transient_lastDdlTime'='1442373214')

------------------------------------------------------------
13 rows selected (0.129 seconds)
0: jdbc:hive2://ha-cluster/default>

Issue Links

• duplicates: HIVE-14013 Describe table doesn't show unicode properly
• is duplicated by: HIVE-11332 Unicode table comments do not work