[HDFS-10932] Ozone : fix XceiverClient slow shutdown

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Hadoop Flags: Reviewed
• Assignee: Chen Liang
• Reporter: Chen Liang
• Created: 29/Sep/16 22:31
• Updated: 26/Apr/18 21:48
• Resolved: 01/Oct/16 00:13

Description

Currently XceiverClient is the underlying entity of DistributedStorageHandler.newKeyWriter() and DistributedStorageHandler.newKeyReader() for making call to container for read/write. When XceiverClient gets closed, group.shutdownGracefully() gets called, which is an asynchronous call.

A problem is that this asynchronous call has default quiet period of 2 seconds before it actually shutdown, so if we have a burst of read/write calls, we would end up having threads created faster than they got terminated, reaching system limit at some point.

Ideally, this needs to be fixed with cached clients instead of creating new thread each time. This JIRA only tries to give a temporary fix for the time being.

Thanks anu for the offline discussion.