[IMPALA-9086] Show Hive configurations in /hadoop-varz page

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.4.0
• Component/s: Frontend
• Labels: ramp-up supportability
• Epic Color: ghx-label-2
• Assignee: Saurabh Katiyal
• Reporter: Quanlong Huang
• Created: 23/Oct/19 03:28
• Updated: 27/May/24 07:35
• Resolved: 01/Mar/24 19:05

Description

Hive configurations are not shown in /hadoop-varz page. It helps to check hive.metastore.warehouse.dir and hive.metastore.warehouse.external.dir used in FE. We should also add the /hadoop-varz page in catalogd WebUI.

The URL callback is registered in be/src/service/impala-http-handler.cc:

  webserver->RegisterUrlCallback("/hadoop-varz", "hadoop-varz.tmpl",
      MakeCallback(this, &ImpalaHttpHandler::HadoopVarzHandler), true);

https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/service/impala-http-handler.cc#L133-L134

The handler, ImpalaHttpHandler::HadoopVarzHandler(), is implemented by calling a JNI method to invoke FE method JniFrontend.getAllHadoopConfigs():
https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L626-L631

We simply create a static object of org.apache.hadoop.conf.Configuration and use it for each /hadoop-varz http request. We can consier changing its type to org.apache.hadoop.hive.conf.HiveConf (inherrits Configuration) which might contain all the hive configurations.

To add the same page for catalogd, we need to register a similar handler in CatalogServer::RegisterWebpages()
https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/catalog/catalog-server.cc#L434
Also add a new method in JniCatalog.java to return the configurations throught JNI, add JNI stuffs in catalog.cc and catalog.h (similar to existing JNI methods, e.g. get_catalog_object_id_).

Issue Links

• is related to: IMPALA-12977 add search and pagination to /hadoop-varz