[HDDS-6577] Configurations to reserve HDDS volume space are not followed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: Ozone Datanode, SCM
• Labels: pull-request-available
• Assignee: Aswin Shakil
• Reporter: Ethan Rose
• Created: 12/Apr/22 19:49
• Updated: 06/Sep/22 17:45
• Resolved: 16/Jun/22 01:34

Description

Ozone currently has two configuration keys that can be used to prevent datanode volumes from being completely filled up:

hdds.datanode.du.reserved: A list of key-value pairs mapping each volume's path to a fixed size that should be left empty on the volume. There is no default value.

hdds.datanode.storage.utilization.critical.threshold: A percentage of each datanode volume that should be left empty. The default value is 95%.

In a live cluster, we have seen volumes fill up past the default 95% value. In a docker based test using a fixed size tmpfs volume mount, the configs also did not seem to be respected. It is also unclear how the configs are reconciled if configured with conflicting values. This Jira aims to provide a reliable way to limit datanode volume usage, and add tests to simulate fixed capacity disks for testing the functionality.

Issue Links

• relates to: HDDS-6901 Configure HDDS volume reserved as percentage of the volume space
• links to: GitHub Pull Request #3484