[KYLIN-4315] Use metadata numRows in beeline client for quick row counting

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v3.1.0
• Component/s: Job Engine
• Sprint: Sprint 51
• Assignee: Congling Xia
• Reporter: Congling Xia
• Created: 26/Dec/19 14:41
• Updated: 31/Jul/20 12:21
• Resolved: 23/May/20 01:21

Description

Hi, I find that in `BeelineHiveClient`, method `getHiveTableRows` uses "select count from <tb_name>" for table row counting. The method is invoked in flat intermediate table redistribution step in cube building.

This stats can be loaded in metastore. It costs much less time than scanning all rows in Hive table. Since intermediate tables are created and inserted by Kylin, statistics will be automatically calculated and stored in metastore when `hive.stats.autogather` is enabled (which is the default setting for Hive). 

ref Hive wiki for more detail about `numRows` stats: https://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables%E2%80%93ANALYZE

Issue Links

• is duplicated by: KYLIN-4509 get hive table rows from metadata when using beeline
• is related to: KYLIN-4526 Enhance get the hive table rows
• links to: GitHub Pull Request #1024