[IMPALA-7164] Define public API for RuntimeProfile

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: resource-management usability
• Epic Link: Improve Impala Profile User Experience
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 12/Jun/18 22:12
• Updated: 18/Oct/18 21:50

Description

Currently the only public API for the runtime profile is the Thrift definition. The exact layout and counters are all implementation-dependent and subject to change. People are trying to build tools that consume runtime profiles and process them, but that's hard since the counters can change from version to version and sometimes the semantics change.

I think we need to figure out which things are part of the public API, validate that they make sense and document them clearly. We should also clearly document that things outside of this public API are subject to change without notice. I don't think the public API necessarily needs to be "porcelain", but we should generally try to avoid unnecessary changes and mention any changes in release notes etc.

We could start simple and just collect "public" counter names in a module and comment each of them.

Issue Links

• is related to: IMPALA-7550 Create documentation for all profile fields