Hey Ming Ma good to hear from you. I should have given a bit more context in the description. Doing it now.
So we are trying to solve two inter-related issues here:
- Inconsistency between the config value of ipc.server.handler.queue.size, which is defined as "How many calls per handler are allowed in the queue.", and the actual queue size when FairCallQueue is used, which lets 4 times more.
- A need to reconfigure the cluster when switching from one CallQueue implementation to another.
The latter is important to prevent NN meltdown. Before we learned about this issue we saw too many clients connecting to the NameNode and making it unresponsive. This is related to read requests like listStatus and getFileInfo. And it worked fine with the default queue, but not with FairCallQueue. Looking deeper it was noticed that NN allowed more requests with FairCallQueue than with the default. We had to reconfigure NN and adjust the queue size, but now we have problem 1. Hope this makes sense.
With your example when all requests go only into the high priority queue one should probably switch to the default queue as FairCallQueue functionality is not utilized.