Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21485

Hive desc operation takes more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3.4
    • Fix Version/s: None
    • Component/s: CLI, Hive

      Description

      Hive desc [formatted|extended] operation cost more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4. This is mainly caused by showing stats for partitioned tables which was introduced by HIVE-16098 when the partitioned tables have a large amount of partitions. In our case, the number of partition is 187221.

      hive> desc bus.kafka_data;
      OK
      id            	        string
      ...
      d                  	    map<string,string>
      stat_date           	string
      log_id              	string
      
      # Partition Information
      # col_name            	data_type           	comment
      
      stat_date           	string
      log_id              	string
      Time taken: 115.342 seconds, Fetched: 42 row(s)
      

      same operation executed in hive-1.2.1 and only cost 2 seconds.

      hive> desc bus.kafka_data;
      OK
      id            	        string
      ...
      d                  	    map<string,string>
      stat_date           	string
      log_id              	string
      
      # Partition Information
      # col_name            	data_type           	comment
      
      stat_date           	string
      log_id              	string
      Time taken: 2.037 seconds, Fetched: 42 row(s)
      
      

        Attachments

        1. HIVE-21485.patch
          5 kB
          Qingxin Wu

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                wuqingxinnn Qingxin Wu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m