[HDDS-8191] Avoid false positive failures in scanners

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Unassigned
• Reporter: Ethan Rose
• Created: 16/Mar/23 21:09
• Updated: 12/Jun/23 23:57
• Resolved: 12/Jun/23 23:57

Description

Sometimes volume or container scan may fail due to external issues in the file system or operation system that do not affect the content of the containers or volumes and are caused by simple admin mistakes. This jira proposes having the following failures be logged by the scanners but not cause their contents to be marked unhealthy:

• Too many open files
  • Can be fixed by the admin adjusting the file descriptor limit for the node.
• Permission denied for reading or writing.
  • Can be fixed by the admin fixing filesystem permissions.
• No space left on device
  • Should be prevented by configuring hdds.datanode.dir.du.reserved.percent to leave some free space on the volume.
  • Failing full volumes could put an already full cluster in a worse state by triggering replication.

Issue Links

• is fixed by: HDDS-7365 Integrate container and volume scanners