Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
RequestHedgingProxyProvider will send its initial request to all NameNodes. It is meant to then detect the active, and send subsequent requests to the active until it fails over.
It does this by remember the last NameNode to fail, but unless the set of NameNodes minus the last to fail is a set of size 1, it will continue to use a thread pool and send multiple requests at a time. If there are 3 or more NameNodes, however, that means a thread pool and multiple requests will always be sent.
The correct logic in the case of more than 2 NameNodes is probably to use the last known active and then when it fails, to reset the state, query all of them in parallell, and then use the one the one that looks like the active.