[HDDS-11512] Create Grafana dashboard for tracking system wide deletion

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Dashboards
• Assignee: Tejaskriya Madhan
• Reporter: Ethan Rose
• Created: 30/Sep/24 21:19
• Updated: 25/Oct/24 06:16

Description

Data deletion in Ozone must flow through three different components asynchronously, and it can be difficult to tell what stage it is in when space has not yet been reclaimed. This Jira is to add a Grafana dashboard for end to end deletion tracking across the whole system in one place. This should include all the metrics described in HDDS-11511, in addition to total cluster space usage from SCM.

For each component, another line should show the aggregate deletion amount across all threads. For example, if there are 3 OM key delete threads and they each deleted 10 keys at a given point in time, this line would show 30 keys deleted at that time.

The dashboard should support specifying a datanode host to focus on the progress of its deletion threads and its space usage per volume.
 

Issue Links

• is required by: HDDS-11514 Set optimal default values for delete configurations based on live cluster testing
• requires: HDDS-11511 OM deletion services should have consistent metrics