[HDDS-1672] Improve locking in OzoneManager

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.4.1
• Component/s: Ozone Manager
• Labels: pull-request-available
• Target Version/s: 0.4.1
• Sprint: Done
• Assignee: Bharat Viswanadham
• Reporter: Bharat Viswanadham
• Created: 12/Jun/19 05:52
• Updated: 05/Aug/19 14:45
• Resolved: 28/Jun/19 21:06

Description

In this Jira, we shall follow the new lock ordering. In this way, in volume requests we can solve the issue of acquire/release/reacquire problem. And few bugs in the current implementation of S3Bucket/Volume operations.

 

Currently after acquiring volume lock, we cannot acquire user lock. 

This is causing an issue in Volume request implementation, acquire/release/reacquire volume lock.

 

Case of Delete Volume Request: 

1. Acquire volume lock.
2. Get Volume Info from DB
3. Release Volume lock. (We are releasing the lock, because while acquiring volume lock, we cannot acquire user lock0
4. Get owner from volume Info read from DB
5. Acquire owner lock
6. Acquire volume lock
7. Do delete logic
8. release volume lock
9. release user lock

 

We can avoid this acquire/release/reacquire lock issue by making volume lock as low weight. 

 

In this way, the above deleteVolume request will change as below

1. Acquire volume lock
2. Get Volume Info from DB
3. Get owner from volume Info read from DB
4. Acquire owner lock
5. Do delete logic
6. release owner lock
7. release volume lock. 

Same issue is seen with SetOwner for Volume request also.

During HDDS-1620 arp brought up this issue. 

I am proposing the above solution to solve this issue. Any other idea/suggestions are welcome.

This also resolves a bug in setOwner for Volume request.

Issue Links

• blocks: HDDS-1689 Implement S3 Create Bucket request to use Cache and DoubleBuffer
• is blocked by: HDDS-1723 Create new OzoneManagerLock class
• links to: GitHub Pull Request #949
• links to: GitHub Pull Request #1016