Details
-
Bug
-
Status: To Do
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
Greetings
I am building a Stellar MAAS machine learning service to process some data. I am concerned about the availability of the deployed service.
The MAAS deploy .sh allows me to define memory allocated as well as the running containers. However, after this, it seems that YARN and Zookeeper takes care of the rest...
Particularly, if one instance of the service is taking a lot of the data from the pipeline, how does YARN or Zookeeper transfers the load? what is the condition for this to happen if this happens?
This in the scenario where I am pushing MAAS machine learning model for threat detection. JMeter tests over the endpoint tells me that the service (on localhost) increases response time from 20ms to 1-2 seconds if there is an excessive load.
How are YARN and Zookeeper making my service available and load-balanced? what is the criteria to do so?
Also, this might be a (me) problem, but on my end, after the MAAS service has been deployed, after several days, Stellar is not able to find the service anymore, even if it still registered as a YARN application. If I restart Zookeeper the services gets back available on Stellar again. Is this a known problem or is this a problem on my end?
Sorry for the lengthy question, but there is not much documentation on this.