[HDDS-7470] [Quota] bucket is created crossing quota makes OM crash and unable to start

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Labels: pull-request-available
• Assignee: Sumit Agrawal
• Reporter: Sumit Agrawal
• Created: 09/Nov/22 11:28
• Updated: 07/Dec/22 02:45
• Resolved: 18/Nov/22 08:51

Description

./ozone sh volume create --space-quota=1GB --namespace-quota=10 /sumit

./ozone sh bucket create --space-quota=800MB --namespace-quota=5 /sumit/one

./ozone sh bucket create --space-quota=800MB --namespace-quota=5 /sumit/two

<-- This step OM is crashed, and on restart, keeps crashing.

The reason is exception is thrown during applyTransaction(), and OM terminates. On restart, again it try apply the transaction and keeps crashing.

 

2022-11-09 16:41:08,317 INFO server.Server (Server.java:doStart(375)) - jetty-9.4.49.v20220914; built: 2022-09-14T01:07:36.601Z; git: 4231a3b2e4cb8548a412a789936d640a97b1aa0a; jvm 1.8.0_292-b10
2022-11-09 16:41:08,345 ERROR ratis.OzoneManagerStateMachine (ExitUtils.java:terminate(133)) - Terminating with exit status 1: Request cmdType: CreateBucket
traceID: ""
clientId: "client-315E9036A608"
userInfo {
userName: "sumitagrawal"
remoteAddress: "127.0.0.1"
hostName: "localhost"
}
version: 3
createBucketRequest {
bucketInfo {
volumeName: "sumit"
bucketName: "two"
acls {
type: USER
name: "sumitagrawal"
rights: "\200"
aclScope: ACCESS
}
...
...
isVersionEnabled: false
storageType: DISK
creationTime: 1667992142575
objectID: 0
updateID: 0
modificationTime: 1667992142575
usedBytes: 0
quotaInBytes: 838860800
quotaInNamespace: 5
usedNamespace: 0
owner: "sumitagrawal"
}
}
failed with exception
java.lang.IllegalArgumentException: Total buckets quota in this volume should not be greater than volume quota : the total space quota is set to:1677721600. But the volume space quota is:1073741824
at org.apache.hadoop.ozone.om.request.bucket.OMBucketCreateRequest.checkQuotaBytesValid(OMBucketCreateRequest.java:408)
at org.apache.hadoop.ozone.om.request.bucket.OMBucketCreateRequest.validateAndUpdateCache(OMBucketCreateRequest.java:210)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:311)
at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.runCommand(OzoneManagerStateMachine.java:533)
at org.apache.hadoop.ozone.om.ratis.OzoneManagerStateMachine.lambda$1(OzoneManagerStateMachine.java:324)
at java.util.concurrent.CompletableFuture$AsyncSupply.run$$$capture(CompletableFuture.java:1604)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2022-11-09 16:41:08,350 INFO om.OzoneManagerStarter (StringUtils.java:lambda$startupShutdownMessage$0(124)) - SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down OzoneManager at sumitagrawal-MBP16/192.168.0.103
************************************************************/

Issue Links

• is related to: HDDS-7497 Ozone fs `mkdir` does not update bucket's `usedNamespace`
• links to: GitHub Pull Request #3960