[IMPALA-9375] Remove DirectMetaProvider usage from CatalogMetaProvider

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.4.0
• Fix Version/s: Impala 4.4.0
• Component/s: Catalog
• Target Version: Impala 4.3.0
• Epic Color: ghx-label-11
• Assignee: Vihang Karajgaonkar
• Reporter: Vihang Karajgaonkar
• Created: 11/Feb/20 21:02
• Updated: 27/Mar/24 19:00
• Resolved: 27/Mar/24 19:00

Description

I see that CatalogMetaProvider uses DirectMetaProvider here https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L239

There are only a couple of places where it is used within CatalogMetaProvider. We should implement those remaining APIs in catalog-v2 mode and remove the usage of DirectMetaProvider from CatalogMetaProvider.

DirectMetaProvider starts by default a MetastoreClientPool (with 10 connections). This is unnecessary given that catalog already makes the connections to HMS at its startup. It also slows down the coordinator startup time if there are HMS connection issues.

Issue Links

• is related to: IMPALA-9609 Minimize Frontend activity in executor only Impalas
• relates to: IMPALA-9405 Unify MetaStoreClientPool Usage