Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.2
    • Component/s: bsp core, messaging, pipes
    • Labels:

      Description

      Adapting Hadoop Pipes to Hama for C++ integration.

      The package org.apache.hama.pipes contains all pipes specific code. The C++ implementation is located in the c++ folder under pipes and utils. There are also some minor modifications in other packages but you can checkout the full Hama source here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/

      The changelog can be found here: here

      Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
      Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/

      Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

      virtual void sendMessage(const string& peerName, const string& msg) = 0;

      Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

      virtual const string& getCurrentMessage() = 0;

      Returns a message from the peer's received messages queue (a FIFO).

      virtual int getNumCurrentMessages() = 0;

      Returns the number of messages in the peer's received messages queue.

      virtual void sync() = 0;

      Barrier Synchronization.
      Sends all the messages in the outgoing message queues to the corresponding remote peers.

      virtual long getSuperstepCount() = 0;

      Returns the count of current super-step

      virtual const string& getPeerName() = 0;

      Returns the name of this peer in the format "hostname:port".

      virtual const string& getPeerName(int index) = 0;

      Returns the name of n-th peer from sorted array by name.

      virtual int getPeerIndex() = 0;

      Returns the index of this peer from sorted array by name.

      virtual vector<string> getAllPeerNames() = 0;

      Returns the names of all the peers executing tasks from the same job (including this peer).

      virtual int getNumPeers() = 0;

      Returns the number of peers

      virtual void clear() = 0;

      Clears all queues entries.

      virtual void write(const string& key, const string& value) = 0;

      Writes a key/value pair to the output collector

      virtual bool readNext(string& key, string& value) = 0;

      Deserializes the next input key value into the given objects;

      virtual void reopenInput() = 0;

      Closes the input and opens it right away, so that the file pointer is at the beginning again.

      1. HAMA-619.patch
        2.78 MB
        Martin Illecker
      2. HAMA-619.patch
        66 kB
        Thomas Jungblut

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Martin Illecker
              Reporter:
              Martin Illecker
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development