When setting the scope of my service to application, the client side will run out of memory after 5-10 thousand hits. The service object on the server side is thread safe and works with both application and request scope. The server does not have any leak.
To fix memory leak, inside the generated stub, I wrap the call with a try/catch/finally, and in that, I put:
This would clear out the memory and no more leak.