Update for the description. Debugged more at this front based on the comments from Enis.
The cause is that for the primary replica, if its retry is exhausted too fast, f.get()  returns ExecutionException. This Exception needs to be ignored and continue with the replicas.
The other issue is that after adding calls for the replicas, if the first completed task gets ExecutionException (due to the retry exhausted), it throws the exception to the client.
In this case, it needs to loop through these tasks, waiting for the success one. If no one succeeds, throw exception.
Similar for the scan as well