Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Not A Problem
-
None
Description
SINGA has a set of message passing interface following ZeroMQ.
Currently, the underlying implementation uses ZeroMQ.
Another commonly used message passing library is MPI.
The comparison between ZeroMQ and MPI is discussed herer http://singa.apache.org/docs/communication.html.
This ticket is going to implement the message passing interface using MPI.
It would be used for optimize the communication between GPUs using RMDA.
Particularly, first we need to implement the intra-process communication by ourselves, e.g., using a global message queue. The Send and Recv functions operate against the global message queue. Second, we need to implement the inter-process communication using MPI, which involves the MPI initialization and message sending and receiving.