[HBASE-20499] Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.5.0
• Component/s: rpc
• Hadoop Flags: Reviewed
• Release Note: Added new config 'hbase.ipc.server.replication.max.callqueue.length'
• Assignee: Nihal Jain
• Reporter: Nihal Jain
• Created: 27/Apr/18 07:18
• Updated: 20/Jul/22 05:49
• Resolved: 19/Jul/22 06:02

Description

In SimpleRpcScheduler's constructor, instead of the using maxQueueLength as default value for replicationExecutor/replicationExecutor's max queue length:

int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH,
handlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
int maxPriorityQueueLength =
conf.getInt(RpcScheduler.IPC_SERVER_PRIORITY_MAX_CALLQUEUE_LENGTH, maxQueueLength);
.
.
this.replicationExecutor = replicationHandlerCount > 0 ? new FastPathBalancedQueueRpcExecutor(
"replication.FPBQ", replicationHandlerCount, RpcExecutor.CALL_QUEUE_TYPE_FIFO_CONF_VALUE,
maxQueueLength, priority, conf, abortable) : null;

See SimpleRpcScheduler

We can do the following:

int maxQueueLength = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH,
handlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
int maxPriorityQueueLength =
conf.getInt(RpcScheduler.IPC_SERVER_PRIORITY_MAX_CALLQUEUE_LENGTH, priorityHandlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
.
.
int maxQueueLengthForReplication = conf.getInt(RpcScheduler.IPC_SERVER_MAX_CALLQUEUE_LENGTH, replicationHandlerCount * RpcServer.DEFAULT_MAX_CALLQUEUE_LENGTH_PER_HANDLER);
.
.
this.replicationExecutor = replicationHandlerCount > 0 ? new FastPathBalancedQueueRpcExecutor(
"replication.FPBQ", replicationHandlerCount, RpcExecutor.CALL_QUEUE_TYPE_FIFO_CONF_VALUE,
maxQueueLengthForReplication , priority, conf, abortable) : null;

 

Also, we can make the maximum replication call queue length configurable, similar to general and priority call queue length.