Adapting Hadoop Pipes to Hama for C++ integration.
The package org.apache.hama.pipes contains all pipes specific code. The C++ implementation is located in the c++ folder under pipes and utils. There are also some minor modifications in other packages but you can checkout the full Hama source here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/
The changelog can be found here: here
Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)
virtual void sendMessage(const string& peerName, const string& msg) = 0;
Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.
virtual const string& getCurrentMessage() = 0;
Returns a message from the peer's received messages queue (a FIFO).
virtual int getNumCurrentMessages() = 0;
Returns the number of messages in the peer's received messages queue.
virtual void sync() = 0;
Sends all the messages in the outgoing message queues to the corresponding remote peers.
virtual long getSuperstepCount() = 0;
Returns the count of current super-step
virtual const string& getPeerName() = 0;
Returns the name of this peer in the format "hostname:port".
virtual const string& getPeerName(int index) = 0;
Returns the name of n-th peer from sorted array by name.
virtual int getPeerIndex() = 0;
Returns the index of this peer from sorted array by name.
virtual vector<string> getAllPeerNames() = 0;
Returns the names of all the peers executing tasks from the same job (including this peer).
virtual int getNumPeers() = 0;
Returns the number of peers
virtual void clear() = 0;
Clears all queues entries.
virtual void write(const string& key, const string& value) = 0;
Writes a key/value pair to the output collector
virtual bool readNext(string& key, string& value) = 0;
Deserializes the next input key value into the given objects;
virtual void reopenInput() = 0;
Closes the input and opens it right away, so that the file pointer is at the beginning again.