Details
-
Improvement
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
My users recently complained 'The NameNode is much slower than before' to me. The reason is the cluster and jobs are getting bigger and bigger. So is the pressure of NameNode. I explained the pressure was heavy so the rpc requests must wait, but they were not satisfied. Because they thought the original quality of the service should be guaranteed. They were never told the NameNode would be so slow and all their services were built based on the assumption that the NameNode would always respond as fast as before.
From the user's standpoint they are right. So my question is how to give the user a guarantee about RPC requests. The natural idea is RPC Quota, just like name quota and space quota. The quota can help users to understand the rpc requests are also a limit resource. And when they apply quota to the administrator, the admin would have the chance to distribute the resource and make a plan for the cluster. e.g. We have 200 quota for addBlock and they are all allocated. Even the peak doesn't reach 200, I should reject other users from applying to reserve the resource. The new user should be mounted to other namespaces.
It's still an initial idea now. I'll think again carefully and make a detailed proposal. All advice are welcome!