[HIVE-23496] Add a flag to disable DB scan during Hive Server startup

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.3.7
• Component/s: HiveServer2
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: David Rabinowitz
• Created: 18/May/20 21:51
• Updated: 26/Jul/20 00:35

Description

At start-up, HiveServer2 will do a full DB & Table scan. For large Hive Metastores, this operation could take 10+ minutes to complete. The goal of this task is to make this “verification” optional if possible.

Currently, Hive 2.3.7 has the following in HiveMaterializedViewsRegistry that is causing this behavior.

 

for (String dbName : db.getAllDatabases()) {
  // TODO: We should enhance metastore API such that it returns only
  // materialized views instead of all tables
  tables.addAll(db.getAllTableObjects(dbName));
}

It seems that <= Hive 2.3.x && >= Hive 3.0 does not have this behavior.

 

Issue Links

• relates to: HIVE-18386 Create dummy materialized views registry and make it configurable
• links to: GitHub Pull Request #1027