[IMPALA-10168] Expose JSON catalog objects in catalogd's debug page

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Labels: observability
• Epic Color: ghx-label-12
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 15/Sep/20 09:49
• Updated: 28/Oct/20 01:04
• Resolved: 28/Oct/20 01:04

Description

Catalogd has a debug web page for showing catalog objects. Example URL: http://localhost:25020/catalog_object?object_type=TABLE&object_name=functional.alltypes

Catalog objects are shown in thrift debug strings which are hard to parse for tests. We do have some tests parsing simple things from the thrift debug strings. For instances, we extract the catalog version here:
https://github.com/apache/impala/blob/fc19e70cbc94735c38e64573c1e015878f218281/tests/common/impala_service.py#L165
We verify whether a table is loaded by checking some substrings:
https://github.com/apache/impala/blob/9ecbe7d3dc22d7233e4bf2f91aaab43d0ad4e373/tests/custom_cluster/test_automatic_invalidation.py#L33-L34

However, it's hard to parse the partition ids of a hdfs table since the debug string is too complex. It'd be better to support exposing JSON results to ease test codes. JSON format results also help for customer to develop their monitor tools.

The new URL can be something like http://localhost:25020/catalog_object?json&object_type=TABLE&object_name=functional.alltypes (Just adding a "json" parameter).