[KAFKA-15158] Add metrics for RemoteDeleteRequestsPerSec, RemoteDeleteErrorsPerSec, BuildRemoteLogAuxStateRequestsPerSec, BuildRemoteLogAuxStateErrorsPerSec

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.7.0
• Labels: tiered-storage
• Assignee: Gantigmaa Selenge
• Reporter: Divij Vaidya
• Created: 06/Jul/23 14:39
• Updated: 21/Dec/23 11:55
• Resolved: 19/Dec/23 07:03

Description

Add the following metrics for better observability into the RemoteLog related activities inside the broker.

1. RemoteWriteRequestsPerSec

2. RemoteDeleteRequestsPerSec

3. BuildRemoteLogAuxStateRequestsPerSec

 

These metrics will be calculated at topic level (we can add them at brokerTopicStats)

RemoteWriteRequestsPerSec will be marked on every call to RemoteLogManager#
copyLogSegmentsToRemote() already covered by KAFKA-14953
 
RemoteDeleteRequestsPerSec will be marked on every call to RemoteLogManager#cleanupExpiredRemoteLogSegments(). This method is introduced in https://github.com/apache/kafka/pull/13561 

BuildRemoteLogAuxStateRequestsPerSec will be marked on every call to ReplicaFetcherTierStateMachine#buildRemoteLogAuxState()

 

(Note: For all the above, add Error metrics as well such as RemoteDeleteErrorPerSec)

(Note: This requires a change in KIP-405 and hence, must be approved by KIP author satishd )
 

Issue Links

• is a child of: KAFKA-15420 Kafka Tiered Storage V1
• links to: GitHub Pull Request #14375
• mentioned in: Page Loading...