[FLINK-28243] Considering only time of targeted operation for calculation benchmarks performance

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Benchmarks
• Assignee: Unassigned
• Reporter: Anton Kalashnikov
• Created: 24/Jun/22 08:30
• Updated: 24/Jun/22 13:31

Description

Right now, in many benchmarks, the iteration time consists of many parts which are not always connected to the target of the particular benchmark. For example, if we want to benchmark async operator, we want to take into account only the time which was spent on this operator but currently, the time of one iteration of asyncOperatorBenchmark is `job/tasks initialization phase` + `operator execution` + `shutdown phase`.

Ideally, we want to isolate somehow `operator execution` time and consider only it in this benchmark but `job/tasks initialization phase` should be benchmarked separately.

Issue Links

• relates to: FLINK-28241 Massive regression on 20.06.2021