Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10168

Expose JSON catalog objects in catalogd's debug page

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.0.0
    • None
    • ghx-label-12

    Description

      Catalogd has a debug web page for showing catalog objects. Example URL: http://localhost:25020/catalog_object?object_type=TABLE&object_name=functional.alltypes

      Catalog objects are shown in thrift debug strings which are hard to parse for tests. We do have some tests parsing simple things from the thrift debug strings. For instances, we extract the catalog version here:
      https://github.com/apache/impala/blob/fc19e70cbc94735c38e64573c1e015878f218281/tests/common/impala_service.py#L165
      We verify whether a table is loaded by checking some substrings:
      https://github.com/apache/impala/blob/9ecbe7d3dc22d7233e4bf2f91aaab43d0ad4e373/tests/custom_cluster/test_automatic_invalidation.py#L33-L34

      However, it's hard to parse the partition ids of a hdfs table since the debug string is too complex. It'd be better to support exposing JSON results to ease test codes. JSON format results also help for customer to develop their monitor tools.

      The new URL can be something like http://localhost:25020/catalog_object?json&object_type=TABLE&object_name=functional.alltypes (Just adding a "json" parameter).

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stigahuang Quanlong Huang
            stigahuang Quanlong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment