[FLINK-34386] Add RocksDB bloom filter metrics

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.18.0
• Fix Version/s: 1.20.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Jufang He
• Reporter: Jufang He
• Created: 06/Feb/24 08:33
• Updated: 20/Feb/24 14:42
• Resolved: 20/Feb/24 14:42

Description

In our production environment, with RocksDB bloom filter enabled, the performance optimization effect on task state reading is obvious. However, there is a lack of usage metrics for bloom filter, If these Metrics are reported via Metrics reporter, it is easy to monitor the effectiveness of bloom filter optimization.

And these metrics are available from RocksDB Statistics:

BLOOM_FILTER_USEFUL: times bloom filter has avoided file reads.

BLOOM_FILTER_FULL_POSITIVE: times bloom FullFilter has not avoided the reads.

BLOOM_FILTER_FULL_TRUE_POSITIVE: times bloom FullFilter has not avoided the reads and data actually exist.

Issue Links

• links to: GitHub Pull Request #24274