[FLINK-33697] FLIP-386: Support adding custom metrics in Recovery Spans

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.19.0
• Component/s: Runtime / Metrics, Runtime / State Backends
• Labels: pull-request-available

Release Note

A braking change has been introduced to the `StateBackend` interface. This is relevant only to users that are implementing their own custom state backends.

Newly added methods `org.apache.flink.runtime.state.StateBackend#createKeyedStateBackend(KeyedStateBackendParameters<K> parameters)` and `org.apache.flink.runtime.state.StateBackend#createOperatorStateBackend(OperatorStateBackendParameters parameters)` have replaced previous versions of the `createKeyedStateBackend` and `createOperatorStateBackend` methods. The new `parameters` POJO classes contain as fields all of the arguments that were passed directly to those methods.

• Assignee: Piotr Nowojski
• Reporter: Piotr Nowojski
• Created: 29/Nov/23 17:02
• Updated: 12/Jan/24 08:40
• Resolved: 12/Jan/24 08:16

Description

Motivation

FLIP-386 is building on top of FLIP-384. The intention here is to add a capability for state backends to attach custom attributes during recovery to recovery spans. For example RocksDBIncrementalRestoreOperation could report both remote download time and time to actually clip/ingest the RocksDB instances after rescaling.

Issue Links

• is related to: FLINK-33696 FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter
• is related to: FLINK-33695 FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces
• links to: GitHub Pull Request #24023