[HIVE-21485] Hive desc operation takes more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.4
• Component/s: CLI, Hive
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Qingxin Wu
• Due: 21/Mar/19
• Created: 21/Mar/19 12:01
• Updated: 16/Jun/20 11:44

Description

Hive desc [formatted|extended] operation cost more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4. This is mainly caused by showing stats for partitioned tables which was introduced by HIVE-16098 when the partitioned tables have a large amount of partitions. In our case, the number of partition is 187221.

hive> desc bus.kafka_data;
OK
id            	        string
...
d                  	    map<string,string>
stat_date           	string
log_id              	string

# Partition Information
# col_name            	data_type           	comment

stat_date           	string
log_id              	string
Time taken: 115.342 seconds, Fetched: 42 row(s)

same operation executed in hive-1.2.1 and only cost 2 seconds.

hive> desc bus.kafka_data;
OK
id            	        string
...
d                  	    map<string,string>
stat_date           	string
log_id              	string

# Partition Information
# col_name            	data_type           	comment

stat_date           	string
log_id              	string
Time taken: 2.037 seconds, Fetched: 42 row(s)

Issue Links

• relates to: HIVE-22453 Describe table unnecessarily fetches partitions
• links to: GitHub Pull Request #577