[FLINK-23399] Add a performance benchmark for statebackend rescaling

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.16.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available stale-assigned
• Assignee: Yanfei Lei
• Reporter: Yanfei Lei
• Created: 15/Jul/21 09:38
• Updated: 14/Jun/22 06:19
• Resolved: 11/Apr/22 08:48

Description

We notice that rescaling is not covered in the current state benchmark, so we'd like to introduce a benchmark to test performance of state backend restore durign rescaling in flink-benchmark.

The benchmark process is:

(1) generate some states,

(2) change the parallelism of the operator, and restore from these states generate before.

The implementation of this benchmark is based on RocksIncrementalCheckpointRescalingTest, and AverageTime is used to measure the rescaling performance on each subtask.

 

And this benchmark does not conflict with `RocksIncrementalCheckpointRescalingBenchmarkTest` in 
PR(#14893). Compare with `RocksIncrementalCheckpointRescalingBenchmarkTest`, this benchmark supports testing rescaling on different state backends, and has a finer granularity.

Issue Links

• is related to: FLINK-21321 Change RocksDB incremental checkpoint re-scaling to use deleteRange
• relates to: FLINK-26560 Make the threshold of the overlap fraction of incremental restoring configurable
• links to: GitHub Pull Request #23
• links to: GitHub Pull Request #19177