[FLINK-32137] Flame graph is hard to use with many task managers

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.1
• Fix Version/s: 1.19.0
• Component/s: Runtime / Web Frontend
• Labels: pull-request-available
• Assignee: Vladimir Matveev
• Reporter: Vladimir Matveev
• Created: 20/May/23 00:48
• Updated: 24/Aug/23 10:17
• Resolved: 23/Aug/23 13:45

Description

In case there are many task managers executing the same operator, the flame graph becomes very hard to use. As you can see on the attached picture, it considers instances of the same lambda function as different classes, and their number seems to be equal to the number of task managers (i.e. each JVM gets its own "class" name, which is expected for lambdas I guess). This lambda function is deep within Flink's own call stack, so this kind of graph is inevitable regardless of the job's own logic, and there is nothing we can do at the job logic's level to fix it.

This behavior makes evaluating the flame graph very hard, because all of the useful information gets "compressed" inside each "column" of the graph, and at the same time, it does not give any useful information since this is just an artifact of the class name generation in the JVM.

Issue Links

• relates to: FLINK-32951 VertexFlameGraphFactoryTest.testLambdaClassNamesCleanUp failed on AZP
• links to: GitHub Pull Request #22854