[HDDS-7479] Investigate -100% Volume and bucket quotas working

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Assignee: Sumit Agrawal
• Reporter: Sumit Agrawal
• Created: 11/Nov/22 12:22
• Updated: 26/Jun/23 08:50
• Resolved: 26/Jun/23 08:50

Description

Need verify quota feature and fix issues:

1. Volume and bucket relationship
2. Bucket with Legacy layout
3. Bucket with FSO layout
4. verify all FS feature for FSO layout related to quota
5. Review code for missing scenario

Issue Links

• is a parent of: HDDS-4257 support quota on key prefix
• is a parent of: HDDS-4565 [FSO]Delete : quotaReleased on directory deletion should consider all sub paths
• is a parent of: HDDS-4249 The usage value needs to be adjusted after client crash
• is a parent of: HDDS-4620 Should remove allocated quota when open key are cleaned up
• is a parent of: HDDS-7346 Cannot set bucket args when the volume has quota set
• is a parent of: HDDS-7598 Quota for snapshot handling
• is a parent of: HDDS-7152 usedBytes and usedNamespace not updated for FSO buckets
• is a parent of: HDDS-4129 Change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE
• is duplicated by: HDDS-7497 Ozone fs `mkdir` does not update bucket's `usedNamespace`
• is related to: HDDS-4273 Volume Namespace:`usedNamespace` works by `ozone sh vol info`
• is related to: HDDS-6650 S3MultipartUpload support update bucket usedNamespace.
• is related to: HDDS-8630 Trash operation doesn't work as expected with Namespace quota set
• is related to: HDDS-8646 [Quota] Ozone key put fails during parallel writes with space quota set

Sub-Tasks

• [Quota] bucket is created crossing quota makes OM crash and unable to start: Resolved
• Bucket property usedNamespace becomes negative after a delete: Resolved
• parallel operation can have failure during commit for quota: Resolved
• Set quota with reduced size than actual used do not fail: Resolved
• local system filename with space is failing when move to ozone: Resolved
• FSO recursive delete directory with hierarchy takes much time for cleanup: Resolved
• Improve setquota, clrquota CLI usage: Resolved
• FSO purge directory for old bucket can update quota for new bucket: Resolved
• namespace quota validation not present multiple places: Resolved
• S3 multipart upload does not update quota namespace for missing parents: Resolved
• S3 multipart upload EC release space quota wrong for old version: Resolved
• Volume Quota not enforced during write when bucket quota is not set.: Resolved
• Restrict change of bucket properties to owner and admins in NativeACL: Resolved
• Volume quota must have all bucket quota set: Resolved
• Deleting a FSO directory does not udpate bucket usedBytes and usedNamespace: Resolved
• setquota should have a check on usedNamespace [NameSpacequota]: Resolved
• Ensure consistent CLI options for storage quota: Resolved
• Minimum Storage quota should be default block size * replication factor: Resolved
• setquota [Volume & Bucker level ] doesn't accepts decimals: Resolved
• Trash operation doesn't work as expected with Namespace quota set: Resolved
• Quota repair counts of namespace and space usages: Resolved
• Quota repair count enable quota feature for old bucket/volume: Resolved