[METRON-1506] When using maas_deploy.sh to list the deployed models, the ApplicationMaster(MaaS) receives 'null request'

• Type: Bug
• Status: To Do
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.4.1
• Labels: easyfix
• Assignee: Unassigned
• Reporter: tantian
• Created: 03/Apr/18 02:38
• Updated: 03/Apr/18 02:42

Description

...

18/03/26 17:16:23 INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_e05_1521078534073_0005_01_000008
18/03/26 17:16:23 INFO impl.ContainerManagementProtocolProxy: Opening proxy : node1:45454
18/03/26 17:16:23 INFO impl.NMClientAsyncImpl: Processing Event EventType: QUERY_CONTAINER for Container container_e05_1521078534073_0005_01_000008
18/03/26 17:16:23 INFO impl.ContainerManagementProtocolProxy: Opening proxy : node1:45454
18/03/26 17:16:32 ERROR service.ApplicationMaster: Received a null request...
18/03/26 17:17:19 INFO service.ApplicationMaster: [ADD]: Received request for model ssss:1.0x1 containers of size 512M at path /user/root/maas/sample
18/03/26 17:17:19 INFO service.ApplicationMaster: Found container id of 5497558138889
18/03/26 17:17:19 INFO callback.LaunchContainer: Setting up container launch container for containerid=container_e05_1521078534073_0005_01_000009
18/03/26 17:17:19 INFO callback.LaunchContainer: Local Directory Contents

...

 

So I read the codes in ModelSubmission.java, and I found the client (maas_deploy.sh) only communicates to the zookeeper, and does not communicate to the ApplicationMaster. But in the code, when the client list the queried deployed models, it sends a null request to the ApplicationMaster, which I thought is not necessary. The related codes are listed here (the red lines are added by me):

ModelRequest request = null;
CuratorFramework client = null;
try {
RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);
client = CuratorFrameworkFactory.newClient(ModelSubmissionOptions.ZK_QUORUM.get(cli), retryPolicy);
client.start();
MaaSConfig config = ConfigUtil.INSTANCE.read(client, ModelSubmissionOptions.ZK_ROOT.get(cli, "/metron/maas/config"), new MaaSConfig(), MaaSConfig.class);
String mode = ModelSubmissionOptions.MODE.get(cli);
if ( mode.equalsIgnoreCase("ADD")) {
request = new ModelRequest() {{
setName(ModelSubmissionOptions.NAME.get(cli));
setAction(Action.ADD);
setVersion(ModelSubmissionOptions.VERSION.get(cli));
setNumInstances(Integer.parseInt(ModelSubmissionOptions.NUM_INSTANCES.get(cli)));
setMemory(Integer.parseInt(ModelSubmissionOptions.MEMORY.get(cli)));
setPath(ModelSubmissionOptions.HDFS_MODEL_PATH.get(cli));
}};
} else if(mode.equalsIgnoreCase("REMOVE")) {
request = new ModelRequest() {{
setName(ModelSubmissionOptions.NAME.get(cli));
setAction(Action.REMOVE);
setNumInstances(Integer.parseInt(ModelSubmissionOptions.NUM_INSTANCES.get(cli)));
setVersion(ModelSubmissionOptions.VERSION.get(cli));
}};
}
else if(mode.equalsIgnoreCase("LIST")) {
String name = ModelSubmissionOptions.NAME.get(cli, null);
String version = ModelSubmissionOptions.VERSION.get(cli, null);
ServiceDiscoverer serviceDiscoverer = new ServiceDiscoverer(client, config.getServiceRoot());

Model model = new Model(name, version);
Map<Model, List<ModelEndpoint>> endpoints = serviceDiscoverer.listEndpoints(model);
for(Map.Entry<Model, List<ModelEndpoint>> kv : endpoints.entrySet()) {
String modelTitle = "Model " + kv.getKey().getName() + " @ " + kv.getKey().getVersion();
System.out.println(modelTitle);
for(ModelEndpoint endpoint : kv.getValue())

{ System.out.println(endpoint); }

}
}

if (ModelSubmissionOptions.LOCAL_MODEL_PATH.has(cli))

{ File localDir = new File(ModelSubmissionOptions.LOCAL_MODEL_PATH.get(cli)); Path hdfsPath = new Path(ModelSubmissionOptions.HDFS_MODEL_PATH.get(cli)); updateHDFS(fs, localDir, hdfsPath); }

if (request != null) {
Queue<ModelRequest> queue = config.createQueue(ImmutableMap.of(ZKQueue.ZK_CLIENT, client));
queue.enqueue(request);
}
} finally {
if (client != null)

{ client.close(); }

}