Attaching updated patch.
Alejandro Abdelnur, Andrew Wang, thanks for the feedback.. think I addressed all your comments. Couple of things though :
prefer to use ArrayLists rather than LinkedLists
Agreed, but since the fill method takes a Queue and since LinkedList implements Queue and not ArrayList, I feel we should stick to LLs.
is using poll() instead, continuing to trigger refills while the return value..
yup that would be cool.. thanks !!
It looks like client requests will only will refill a single key. It seems ..
So, this is how it is now : if Queue doesn't exist for key (or if key has been dropped), the next call from client will be a synchronous call to fill numInitValues which can be different from numValues.
Now, If the client requests n values and there are < n in the queue, ONLY those values will be drained via poll() in the synchronous call and sent to client which is why I named the method getAtMost. My thinking is that on the server side, if the cache size is large, and if the client gets only 1 Key from a request, and if the client requires more, it can fire a second req.. and there is a high probability that the next request from the client for the same Q would return the complete batch (we can tune the refill task to be more aggressive based on e2e testing). Also returning only 1 will kind of guarantee that the server, if bombarded by concurrent requests won't konk off.