Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.2
    • Component/s: bsp core, messaging, pipes
    • Labels:

      Description

      Adapting Hadoop Pipes to Hama for C++ integration.

      The package org.apache.hama.pipes contains all pipes specific code. The C++ implementation is located in the c++ folder under pipes and utils. There are also some minor modifications in other packages but you can checkout the full Hama source here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/

      The changelog can be found here: here

      Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
      Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/

      Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

      virtual void sendMessage(const string& peerName, const string& msg) = 0;

      Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

      virtual const string& getCurrentMessage() = 0;

      Returns a message from the peer's received messages queue (a FIFO).

      virtual int getNumCurrentMessages() = 0;

      Returns the number of messages in the peer's received messages queue.

      virtual void sync() = 0;

      Barrier Synchronization.
      Sends all the messages in the outgoing message queues to the corresponding remote peers.

      virtual long getSuperstepCount() = 0;

      Returns the count of current super-step

      virtual const string& getPeerName() = 0;

      Returns the name of this peer in the format "hostname:port".

      virtual const string& getPeerName(int index) = 0;

      Returns the name of n-th peer from sorted array by name.

      virtual int getPeerIndex() = 0;

      Returns the index of this peer from sorted array by name.

      virtual vector<string> getAllPeerNames() = 0;

      Returns the names of all the peers executing tasks from the same job (including this peer).

      virtual int getNumPeers() = 0;

      Returns the number of peers

      virtual void clear() = 0;

      Clears all queues entries.

      virtual void write(const string& key, const string& value) = 0;

      Writes a key/value pair to the output collector

      virtual bool readNext(string& key, string& value) = 0;

      Deserializes the next input key value into the given objects;

      virtual void reopenInput() = 0;

      Closes the input and opens it right away, so that the file pointer is at the beginning again.

        Attachments

        1. HAMA-619.patch
          2.78 MB
          Martin Illecker
        2. HAMA-619.patch
          66 kB
          Thomas Jungblut

          Issue Links

            Activity

              People

              • Assignee:
                bafu Martin Illecker
                Reporter:
                bafu Martin Illecker
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: