Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.2
    • Component/s: bsp core, messaging, pipes
    • Labels:

      Description

      Adapting Hadoop Pipes to Hama for C++ integration.

      The package org.apache.hama.pipes contains all pipes specific code. The C++ implementation is located in the c++ folder under pipes and utils. There are also some minor modifications in other packages but you can checkout the full Hama source here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/

      The changelog can be found here: here

      Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
      Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/

      Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

      virtual void sendMessage(const string& peerName, const string& msg) = 0;

      Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

      virtual const string& getCurrentMessage() = 0;

      Returns a message from the peer's received messages queue (a FIFO).

      virtual int getNumCurrentMessages() = 0;

      Returns the number of messages in the peer's received messages queue.

      virtual void sync() = 0;

      Barrier Synchronization.
      Sends all the messages in the outgoing message queues to the corresponding remote peers.

      virtual long getSuperstepCount() = 0;

      Returns the count of current super-step

      virtual const string& getPeerName() = 0;

      Returns the name of this peer in the format "hostname:port".

      virtual const string& getPeerName(int index) = 0;

      Returns the name of n-th peer from sorted array by name.

      virtual int getPeerIndex() = 0;

      Returns the index of this peer from sorted array by name.

      virtual vector<string> getAllPeerNames() = 0;

      Returns the names of all the peers executing tasks from the same job (including this peer).

      virtual int getNumPeers() = 0;

      Returns the number of peers

      virtual void clear() = 0;

      Clears all queues entries.

      virtual void write(const string& key, const string& value) = 0;

      Writes a key/value pair to the output collector

      virtual bool readNext(string& key, string& value) = 0;

      Deserializes the next input key value into the given objects;

      virtual void reopenInput() = 0;

      Closes the input and opens it right away, so that the file pointer is at the beginning again.

      1. HAMA-619.patch
        66 kB
        Thomas Jungblut
      2. HAMA-619.patch
        2.78 MB
        Martin Illecker

        Issue Links

          Activity

          Martin Illecker created issue -
          Thomas Jungblut made changes -
          Field Original Value New Value
          Link This issue is related to HAMA-601 [ HAMA-601 ]
          Edward J. Yoon made changes -
          Assignee Martin Illecker [ bafu ]
          Martin Illecker made changes -
          Description Adapting Hadoop Pipes to Hama for C++ integration. *Adapting Hadoop Pipes to Hama for C++ integration.*

          The SVN source is available here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/
          Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
          Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          Martin Illecker made changes -
          Description *Adapting Hadoop Pipes to Hama for C++ integration.*

          The SVN source is available here: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/
          Hama Pipes: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/
          Hama Pipes C++: https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          *Adapting Hadoop Pipes to Hama for C++ integration.*

          Today I finished the first version of Hama Pipes. :-)
          I have added the package *org.apache.hama.pipes* which contains all pipes specific code. The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also some minor modifications in other packages but you can checkout the full Hama source here:
          [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

          The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].

          Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
          Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          Martin Illecker made changes -
          Description *Adapting Hadoop Pipes to Hama for C++ integration.*

          Today I finished the first version of Hama Pipes. :-)
          I have added the package *org.apache.hama.pipes* which contains all pipes specific code. The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also some minor modifications in other packages but you can checkout the full Hama source here:
          [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

          The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].

          Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
          Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          *Adapting Hadoop Pipes to Hama for C++ integration.*

          Today I finished the first version of Hama Pipes. :-)
          I have added the package *org.apache.hama.pipes* which contains all pipes specific code. The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also some minor modifications in other packages but you can checkout the full Hama source here: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

          The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].

          Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
          Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          Hide
          Thomas Jungblut added a comment -

          Thanks, will take the time to review and make suggestions on the weekend.

          Do you have any plans how we can automatically test this?

          Show
          Thomas Jungblut added a comment - Thanks, will take the time to review and make suggestions on the weekend. Do you have any plans how we can automatically test this?
          Hide
          Thomas Jungblut added a comment -

          Lookgs very good to me. What are the next steps we need to plan on that?

          Show
          Thomas Jungblut added a comment - Lookgs very good to me. What are the next steps we need to plan on that?
          Hide
          Martin Illecker added a comment -

          Thanks!
          Currently I'm trying to run some examples, including your MatrixMultiplicationBSP and KMeansBSP.
          After that I will do some performance analysis on these examples.
          You can find the examples here: https://svn.illecker.at/hadoop/applications/bsp/

          Regarding to automatically test, I think we can test the protocol easily. (Request -> Response for each protocol function)
          One test class should be implemented in C++ including the HADOOP_ASSERT construct and one Java test class, to test both sides.

          Show
          Martin Illecker added a comment - Thanks! Currently I'm trying to run some examples, including your MatrixMultiplicationBSP and KMeansBSP. After that I will do some performance analysis on these examples. You can find the examples here: https://svn.illecker.at/hadoop/applications/bsp/ Regarding to automatically test, I think we can test the protocol easily. (Request -> Response for each protocol function) One test class should be implemented in C++ including the HADOOP_ASSERT construct and one Java test class, to test both sides.
          Hide
          Thomas Jungblut added a comment -

          Yep that was what I thought als well.

          So do you think we can assemble a patch and transfer that to the hama trunk?
          Would like to start with streaming in the next few days and need the protocol.

          For the testing, if you don't want to provide a testcase I would like to add a follow-up jira for that.

          Show
          Thomas Jungblut added a comment - Yep that was what I thought als well. So do you think we can assemble a patch and transfer that to the hama trunk? Would like to start with streaming in the next few days and need the protocol. For the testing, if you don't want to provide a testcase I would like to add a follow-up jira for that.
          Hide
          Martin Illecker added a comment - - edited

          I think we should wait until I got the examples working.
          Maybe there are still some minor bugs in it.
          I hope, I will finish with the examples next week!

          We can add a follow-up in jira, but I can implement these test cases later!

          Show
          Martin Illecker added a comment - - edited I think we should wait until I got the examples working. Maybe there are still some minor bugs in it. I hope, I will finish with the examples next week! We can add a follow-up in jira, but I can implement these test cases later!
          Hide
          Thomas Jungblut added a comment -

          Take your time. I can operate on your trunk to add the streaming

          Show
          Thomas Jungblut added a comment - Take your time. I can operate on your trunk to add the streaming
          Hide
          Edward J. Yoon added a comment -

          Cool!

          Show
          Edward J. Yoon added a comment - Cool!
          Martin Illecker made changes -
          Description *Adapting Hadoop Pipes to Hama for C++ integration.*

          Today I finished the first version of Hama Pipes. :-)
          I have added the package *org.apache.hama.pipes* which contains all pipes specific code. The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also some minor modifications in other packages but you can checkout the full Hama source here: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

          The working *revision is 32*, for the changelog see [here|http://hadoop.illecker.at/?page_id=6].

          Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
          Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          *Adapting Hadoop Pipes to Hama for C++ integration.*

          The package *org.apache.hama.pipes* contains all pipes specific code. The C++ implementation is located in the *c++* folder under *pipes* and *utils*. There are also some minor modifications in other packages but you can checkout the full Hama source here: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/]

          The changelog can be found here: [here|http://hadoop.illecker.at/?page_id=6]

          Hama Pipes: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/core/src/main/java/org/apache/hama/pipes/]
          Hama Pipes C++: [https://svn.illecker.at/hadoop/hama-0.5.0-gpu/c++/pipes/]


          Hama Pipes provides the following methods for C++ integration: (similar to Hama BSP)

          *{{virtual void sendMessage(const string& peerName, const string& msg) = 0;}}*
          bq. Send a data with a tag to another BSPSlave corresponding to hostname. Messages sent by this method are not guaranteed to be received in a sent order.

          *{{virtual const string& getCurrentMessage() = 0;}}*
          bq. Returns a message from the peer's received messages queue (a FIFO).

          *{{virtual int getNumCurrentMessages() = 0;}}*
          bq. Returns the number of messages in the peer's received messages queue.

          *{{virtual void sync() = 0;}}*
          {quote}
          Barrier Synchronization.
          Sends all the messages in the outgoing message queues to the corresponding remote peers.
          {quote}

          *{{virtual long getSuperstepCount() = 0;}}*
          bq. Returns the count of current super-step
               
          *{{virtual const string& getPeerName() = 0;}}*
          bq. Returns the name of this peer in the format "hostname:port".
              
          *{{virtual const string& getPeerName(int index) = 0;}}*
          bq. Returns the name of n-th peer from sorted array by name.
              
          *{{virtual int getPeerIndex() = 0;}}*
          bq. Returns the index of this peer from sorted array by name.

          *{{virtual vector<string> getAllPeerNames() = 0;}}*
          bq. Returns the names of all the peers executing tasks from the same job (including this peer).
              
          *{{virtual int getNumPeers() = 0;}}*
          bq. Returns the number of peers
              
          *{{virtual void clear() = 0;}}*
          bq. Clears all queues entries.
              
          *{{virtual void write(const string& key, const string& value) = 0;}}*
          bq. Writes a key/value pair to the output collector
              
          *{{virtual bool readNext(string& key, string& value) = 0;}}*
          bq. Deserializes the next input key value into the given objects;

          *{{virtual void reopenInput() = 0;}}*
          bq. Closes the input and opens it right away, so that the file pointer is at the beginning again.
          Hide
          Thomas Jungblut added a comment -

          Hi Martin, can you please transfer all your documentation to the wiki? The issue will be closed and then gone and I think the wiki is more appropriate to do these documentations.

          http://wiki.apache.org/hama/

          Let me know if you need special rights, normally just registering and writing works.

          Show
          Thomas Jungblut added a comment - Hi Martin, can you please transfer all your documentation to the wiki? The issue will be closed and then gone and I think the wiki is more appropriate to do these documentations. http://wiki.apache.org/hama/ Let me know if you need special rights, normally just registering and writing works.
          Hide
          Martin Illecker added a comment -

          Today I have fixed some bugs of Hama Pipes.
          No problem, I will transfer the documentation to the wiki.

          Show
          Martin Illecker added a comment - Today I have fixed some bugs of Hama Pipes. No problem, I will transfer the documentation to the wiki.
          Hide
          Thomas Jungblut added a comment -

          Hey Martin,

          what's your progress on that? Can we merge it into our trunk?

          Show
          Thomas Jungblut added a comment - Hey Martin, what's your progress on that? Can we merge it into our trunk?
          Hide
          Thomas Jungblut added a comment -

          Hey Martin, I just generated a patch for the java part. Would like to start with streaming soon, so I need to finalize the protocol.

          I have a question about the implementation of the read and write from and to sequence files.

          Why aren't you using the plain "WRITE_KEYVALUE" method?

          You can configure the job to output sequence files, there is no need to implement the stuff on your own.
          Personally I would delete this, but I don't know what parts of your implementations are grounding on it.

          Show
          Thomas Jungblut added a comment - Hey Martin, I just generated a patch for the java part. Would like to start with streaming soon, so I need to finalize the protocol. I have a question about the implementation of the read and write from and to sequence files. Why aren't you using the plain "WRITE_KEYVALUE" method? You can configure the job to output sequence files, there is no need to implement the stuff on your own. Personally I would delete this, but I don't know what parts of your implementations are grounding on it.
          Thomas Jungblut made changes -
          Attachment HAMA-619.patch [ 12544664 ]
          Hide
          Martin Illecker added a comment -

          Sorry for the delay.
          Thanks for the advice, but I had a problem with reading from a sequence file.
          I know the possibility to write a sequence file via job configuration.
          I will have a look!

          Show
          Martin Illecker added a comment - Sorry for the delay. Thanks for the advice, but I had a problem with reading from a sequence file. I know the possibility to write a sequence file via job configuration. I will have a look!
          Hide
          Thomas Jungblut added a comment -

          No problem, I will subclass the binary protocol and do the serialization in text for streaming. Will let you know if anything changes.

          Show
          Thomas Jungblut added a comment - No problem, I will subclass the binary protocol and do the serialization in text for streaming. Will let you know if anything changes.
          Edward J. Yoon made changes -
          Fix Version/s 0.7.0 [ 12320349 ]
          Hide
          Edward J. Yoon added a comment -

          Let's add this to 0.7 release. If you're busy, I can finish this.

          Show
          Edward J. Yoon added a comment - Let's add this to 0.7 release. If you're busy, I can finish this.
          Edward J. Yoon made changes -
          Fix Version/s 0.7.0 [ 12320349 ]
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577506 ]
          Hide
          Martin Illecker added a comment - - edited

          Sorry for the delay.
          I have created a patch, please let me know if there is something missing!

          Show
          Martin Illecker added a comment - - edited Sorry for the delay. I have created a patch, please let me know if there is something missing!
          Hide
          Edward J. Yoon added a comment -

          Thanks Martin,

          Let's merge this to Hama 0.7 TRUNK. I can review tomorrow.

          Show
          Edward J. Yoon added a comment - Thanks Martin, Let's merge this to Hama 0.7 TRUNK. I can review tomorrow.
          Edward J. Yoon made changes -
          Fix Version/s 0.7.0 [ 12320349 ]
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577506 ]
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577509 ]
          Hide
          Edward J. Yoon added a comment -

          The patch seems outdated. Can you reattach patch for latest trunk version?

          Show
          Edward J. Yoon added a comment - The patch seems outdated. Can you reattach patch for latest trunk version?
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577509 ]
          Hide
          Martin Illecker added a comment -

          Sorry, patch has been updated!

          Show
          Martin Illecker added a comment - Sorry, patch has been updated!
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577526 ]
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577526 ]
          Martin Illecker made changes -
          Attachment HAMA-619.patch [ 12577534 ]
          Hide
          Edward J. Yoon added a comment -

          Thanks! Patch looks great, I'll test tomorrow.

          Show
          Edward J. Yoon added a comment - Thanks! Patch looks great, I'll test tomorrow.
          Hide
          Edward J. Yoon added a comment -

          It works well.

          13/04/09 09:49:41 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
          13/04/09 09:49:42 WARN bsp.BSPJobClient: No job jar file set.  User classes may not be found. See BSPJob#setJar(String) or check Your jar file.
          13/04/09 09:49:42 INFO bsp.BSPJobClient: Running job: job_201304090944_0004
          13/04/09 09:49:45 INFO bsp.BSPJobClient: Job complete: job_201304090944_0004
          13/04/09 09:49:45 INFO bsp.BSPJobClient: The total number of supersteps: 1
          13/04/09 09:49:45 INFO bsp.BSPJobClient: Counters: 5
          13/04/09 09:49:45 INFO bsp.BSPJobClient:   org.apache.hama.bsp.JobInProgress$JobCounter
          13/04/09 09:49:45 INFO bsp.BSPJobClient:     LAUNCHED_TASKS=2
          13/04/09 09:49:45 INFO bsp.BSPJobClient:   org.apache.hama.bsp.BSPPeerImpl$PeerCounter
          13/04/09 09:49:45 INFO bsp.BSPJobClient:     SUPERSTEP_SUM=2
          13/04/09 09:49:45 INFO bsp.BSPJobClient:     TIME_IN_SYNC_MS=143
          13/04/09 09:49:45 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_SENT=2
          13/04/09 09:49:45 INFO bsp.BSPJobClient:     TOTAL_MESSAGES_RECEIVED=2
          

          +1 Thanks, Martin. I've committed this to TRUNK.

          Side notes:

          • If the number of tasks is exceeded allowed limit, it seems hang forever.
          • NullPointerExceptions when the binary path is wrong.
          • Output path is always local filesystem.

          Let's fix later.

          Show
          Edward J. Yoon added a comment - It works well. 13/04/09 09:49:41 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 13/04/09 09:49:42 WARN bsp.BSPJobClient: No job jar file set. User classes may not be found. See BSPJob#setJar( String ) or check Your jar file. 13/04/09 09:49:42 INFO bsp.BSPJobClient: Running job: job_201304090944_0004 13/04/09 09:49:45 INFO bsp.BSPJobClient: Job complete: job_201304090944_0004 13/04/09 09:49:45 INFO bsp.BSPJobClient: The total number of supersteps: 1 13/04/09 09:49:45 INFO bsp.BSPJobClient: Counters: 5 13/04/09 09:49:45 INFO bsp.BSPJobClient: org.apache.hama.bsp.JobInProgress$JobCounter 13/04/09 09:49:45 INFO bsp.BSPJobClient: LAUNCHED_TASKS=2 13/04/09 09:49:45 INFO bsp.BSPJobClient: org.apache.hama.bsp.BSPPeerImpl$PeerCounter 13/04/09 09:49:45 INFO bsp.BSPJobClient: SUPERSTEP_SUM=2 13/04/09 09:49:45 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=143 13/04/09 09:49:45 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=2 13/04/09 09:49:45 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=2 +1 Thanks, Martin. I've committed this to TRUNK. Side notes: If the number of tasks is exceeded allowed limit, it seems hang forever. NullPointerExceptions when the binary path is wrong. Output path is always local filesystem. Let's fix later.
          Hide
          Edward J. Yoon added a comment -

          And, I've changed wiki (added installation guide and line #include <cstdlib>) slightly - http://wiki.apache.org/hama/HamaPipes

          Show
          Edward J. Yoon added a comment - And, I've changed wiki (added installation guide and line #include <cstdlib>) slightly - http://wiki.apache.org/hama/HamaPipes
          Hide
          Edward J. Yoon added a comment -

          Fixed.

          Show
          Edward J. Yoon added a comment - Fixed.
          Edward J. Yoon made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hama trunk #115 (See https://builds.apache.org/job/Hama%20trunk/115/)
          HAMA-619: Hama Pipes (Revision 1465852)

          Result = SUCCESS
          edwardyoon :
          Files :

          • /hama/trunk/CHANGES.txt
          • /hama/trunk/bin/hama
          • /hama/trunk/c++
          • /hama/trunk/c++/pipes
          • /hama/trunk/c++/pipes/.autom4te.cfg
          • /hama/trunk/c++/pipes/Makefile.am
          • /hama/trunk/c++/pipes/Makefile.in
          • /hama/trunk/c++/pipes/aclocal.m4
          • /hama/trunk/c++/pipes/api
          • /hama/trunk/c++/pipes/api/hama
          • /hama/trunk/c++/pipes/api/hama/Pipes.hh
          • /hama/trunk/c++/pipes/api/hama/TemplateFactory.hh
          • /hama/trunk/c++/pipes/compile
          • /hama/trunk/c++/pipes/config.guess
          • /hama/trunk/c++/pipes/config.sub
          • /hama/trunk/c++/pipes/configure
          • /hama/trunk/c++/pipes/configure.ac
          • /hama/trunk/c++/pipes/debug
          • /hama/trunk/c++/pipes/debug/pipes-default-gdb-commands.txt
          • /hama/trunk/c++/pipes/debug/pipes-default-script
          • /hama/trunk/c++/pipes/depcomp
          • /hama/trunk/c++/pipes/impl
          • /hama/trunk/c++/pipes/impl/HamaPipes.cc
          • /hama/trunk/c++/pipes/impl/config.h.in
          • /hama/trunk/c++/pipes/install-sh
          • /hama/trunk/c++/pipes/ltmain.sh
          • /hama/trunk/c++/pipes/missing
          • /hama/trunk/c++/utils
          • /hama/trunk/c++/utils/.autom4te.cfg
          • /hama/trunk/c++/utils/Makefile.am
          • /hama/trunk/c++/utils/Makefile.in
          • /hama/trunk/c++/utils/aclocal.m4
          • /hama/trunk/c++/utils/api
          • /hama/trunk/c++/utils/api/hadoop
          • /hama/trunk/c++/utils/api/hadoop/SerialUtils.hh
          • /hama/trunk/c++/utils/api/hadoop/Splitter.hh
          • /hama/trunk/c++/utils/api/hadoop/StringUtils.hh
          • /hama/trunk/c++/utils/config.guess
          • /hama/trunk/c++/utils/config.sub
          • /hama/trunk/c++/utils/configure
          • /hama/trunk/c++/utils/configure.ac
          • /hama/trunk/c++/utils/depcomp
          • /hama/trunk/c++/utils/impl
          • /hama/trunk/c++/utils/impl/SerialUtils.cc
          • /hama/trunk/c++/utils/impl/Splitter.cc
          • /hama/trunk/c++/utils/impl/StringUtils.cc
          • /hama/trunk/c++/utils/impl/config.h.in
          • /hama/trunk/c++/utils/install-sh
          • /hama/trunk/c++/utils/ltmain.sh
          • /hama/trunk/c++/utils/m4
          • /hama/trunk/c++/utils/m4/hadoop_utils.m4
          • /hama/trunk/c++/utils/missing
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJob.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJobClient.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPTask.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueLineRecordReader.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueTextInputFormat.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/TaskLog.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/Application.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/BinaryProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/DownwardProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplicable.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplication.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesBSP.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesPartitioner.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/StreamingProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/Submitter.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/DownwardProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/MessageType.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/UplinkReader.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/DistributedCacheUtil.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/SequenceFileDumper.java
          • /hama/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hama trunk #115 (See https://builds.apache.org/job/Hama%20trunk/115/ ) HAMA-619 : Hama Pipes (Revision 1465852) Result = SUCCESS edwardyoon : Files : /hama/trunk/CHANGES.txt /hama/trunk/bin/hama /hama/trunk/c++ /hama/trunk/c++/pipes /hama/trunk/c++/pipes/.autom4te.cfg /hama/trunk/c++/pipes/Makefile.am /hama/trunk/c++/pipes/Makefile.in /hama/trunk/c++/pipes/aclocal.m4 /hama/trunk/c++/pipes/api /hama/trunk/c++/pipes/api/hama /hama/trunk/c++/pipes/api/hama/Pipes.hh /hama/trunk/c++/pipes/api/hama/TemplateFactory.hh /hama/trunk/c++/pipes/compile /hama/trunk/c++/pipes/config.guess /hama/trunk/c++/pipes/config.sub /hama/trunk/c++/pipes/configure /hama/trunk/c++/pipes/configure.ac /hama/trunk/c++/pipes/debug /hama/trunk/c++/pipes/debug/pipes-default-gdb-commands.txt /hama/trunk/c++/pipes/debug/pipes-default-script /hama/trunk/c++/pipes/depcomp /hama/trunk/c++/pipes/impl /hama/trunk/c++/pipes/impl/HamaPipes.cc /hama/trunk/c++/pipes/impl/config.h.in /hama/trunk/c++/pipes/install-sh /hama/trunk/c++/pipes/ltmain.sh /hama/trunk/c++/pipes/missing /hama/trunk/c++/utils /hama/trunk/c++/utils/.autom4te.cfg /hama/trunk/c++/utils/Makefile.am /hama/trunk/c++/utils/Makefile.in /hama/trunk/c++/utils/aclocal.m4 /hama/trunk/c++/utils/api /hama/trunk/c++/utils/api/hadoop /hama/trunk/c++/utils/api/hadoop/SerialUtils.hh /hama/trunk/c++/utils/api/hadoop/Splitter.hh /hama/trunk/c++/utils/api/hadoop/StringUtils.hh /hama/trunk/c++/utils/config.guess /hama/trunk/c++/utils/config.sub /hama/trunk/c++/utils/configure /hama/trunk/c++/utils/configure.ac /hama/trunk/c++/utils/depcomp /hama/trunk/c++/utils/impl /hama/trunk/c++/utils/impl/SerialUtils.cc /hama/trunk/c++/utils/impl/Splitter.cc /hama/trunk/c++/utils/impl/StringUtils.cc /hama/trunk/c++/utils/impl/config.h.in /hama/trunk/c++/utils/install-sh /hama/trunk/c++/utils/ltmain.sh /hama/trunk/c++/utils/m4 /hama/trunk/c++/utils/m4/hadoop_utils.m4 /hama/trunk/c++/utils/missing /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJob.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJobClient.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPTask.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueLineRecordReader.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueTextInputFormat.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/TaskLog.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/Application.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/BinaryProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/DownwardProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplicable.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplication.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesBSP.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesPartitioner.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/StreamingProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/Submitter.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/DownwardProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/MessageType.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/UplinkReader.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/util /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/DistributedCacheUtil.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/SequenceFileDumper.java /hama/trunk/pom.xml
          Hide
          Martin Illecker added a comment -

          Thanks Edward for the successful merge!
          The following folders and files should be on the svn:ignore list:

          Folders
          ? c++/install
          ? c++/pipes/impl/.deps
          ? c++/utils/impl/.deps

          Files
          ? c++/pipes/Makefile
          ? c++/pipes/config.log
          ? c++/pipes/config.status
          ? c++/pipes/libtool
          ? c++/pipes/impl/stamp-h1
          ? c++/pipes/impl/config.h
          ? c++/pipes/impl/.dirstamp

          ? c++/utils/libtool
          ? c++/utils/Makefile
          ? c++/utils/config.log
          ? c++/utils/config.status
          ? c++/utils/impl/stamp-h1
          ? c++/utils/impl/config.h
          ? c++/utils/impl/.dirstamp

          Show
          Martin Illecker added a comment - Thanks Edward for the successful merge! The following folders and files should be on the svn:ignore list: Folders ? c++/install ? c++/pipes/impl/.deps ? c++/utils/impl/.deps Files ? c++/pipes/Makefile ? c++/pipes/config.log ? c++/pipes/config.status ? c++/pipes/libtool ? c++/pipes/impl/stamp-h1 ? c++/pipes/impl/config.h ? c++/pipes/impl/.dirstamp ? c++/utils/libtool ? c++/utils/Makefile ? c++/utils/config.log ? c++/utils/config.status ? c++/utils/impl/stamp-h1 ? c++/utils/impl/config.h ? c++/utils/impl/.dirstamp
          Hide
          Edward J. Yoon added a comment -

          Thanks done! Enjoy.

          Show
          Edward J. Yoon added a comment - Thanks done! Enjoy.
          Hide
          Hudson added a comment -

          Integrated in Hama-Nightly #880 (See https://builds.apache.org/job/Hama-Nightly/880/)
          HAMA-619: Hama Pipes (Revision 1465852)

          Result = SUCCESS
          edwardyoon :
          Files :

          • /hama/trunk/CHANGES.txt
          • /hama/trunk/bin/hama
          • /hama/trunk/c++
          • /hama/trunk/c++/pipes
          • /hama/trunk/c++/pipes/.autom4te.cfg
          • /hama/trunk/c++/pipes/Makefile.am
          • /hama/trunk/c++/pipes/Makefile.in
          • /hama/trunk/c++/pipes/aclocal.m4
          • /hama/trunk/c++/pipes/api
          • /hama/trunk/c++/pipes/api/hama
          • /hama/trunk/c++/pipes/api/hama/Pipes.hh
          • /hama/trunk/c++/pipes/api/hama/TemplateFactory.hh
          • /hama/trunk/c++/pipes/compile
          • /hama/trunk/c++/pipes/config.guess
          • /hama/trunk/c++/pipes/config.sub
          • /hama/trunk/c++/pipes/configure
          • /hama/trunk/c++/pipes/configure.ac
          • /hama/trunk/c++/pipes/debug
          • /hama/trunk/c++/pipes/debug/pipes-default-gdb-commands.txt
          • /hama/trunk/c++/pipes/debug/pipes-default-script
          • /hama/trunk/c++/pipes/depcomp
          • /hama/trunk/c++/pipes/impl
          • /hama/trunk/c++/pipes/impl/HamaPipes.cc
          • /hama/trunk/c++/pipes/impl/config.h.in
          • /hama/trunk/c++/pipes/install-sh
          • /hama/trunk/c++/pipes/ltmain.sh
          • /hama/trunk/c++/pipes/missing
          • /hama/trunk/c++/utils
          • /hama/trunk/c++/utils/.autom4te.cfg
          • /hama/trunk/c++/utils/Makefile.am
          • /hama/trunk/c++/utils/Makefile.in
          • /hama/trunk/c++/utils/aclocal.m4
          • /hama/trunk/c++/utils/api
          • /hama/trunk/c++/utils/api/hadoop
          • /hama/trunk/c++/utils/api/hadoop/SerialUtils.hh
          • /hama/trunk/c++/utils/api/hadoop/Splitter.hh
          • /hama/trunk/c++/utils/api/hadoop/StringUtils.hh
          • /hama/trunk/c++/utils/config.guess
          • /hama/trunk/c++/utils/config.sub
          • /hama/trunk/c++/utils/configure
          • /hama/trunk/c++/utils/configure.ac
          • /hama/trunk/c++/utils/depcomp
          • /hama/trunk/c++/utils/impl
          • /hama/trunk/c++/utils/impl/SerialUtils.cc
          • /hama/trunk/c++/utils/impl/Splitter.cc
          • /hama/trunk/c++/utils/impl/StringUtils.cc
          • /hama/trunk/c++/utils/impl/config.h.in
          • /hama/trunk/c++/utils/install-sh
          • /hama/trunk/c++/utils/ltmain.sh
          • /hama/trunk/c++/utils/m4
          • /hama/trunk/c++/utils/m4/hadoop_utils.m4
          • /hama/trunk/c++/utils/missing
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJob.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJobClient.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPTask.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueLineRecordReader.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueTextInputFormat.java
          • /hama/trunk/core/src/main/java/org/apache/hama/bsp/TaskLog.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/Application.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/BinaryProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/DownwardProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplicable.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplication.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesBSP.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesPartitioner.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/StreamingProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/Submitter.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/DownwardProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/MessageType.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/UplinkReader.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/DistributedCacheUtil.java
          • /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/SequenceFileDumper.java
          • /hama/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hama-Nightly #880 (See https://builds.apache.org/job/Hama-Nightly/880/ ) HAMA-619 : Hama Pipes (Revision 1465852) Result = SUCCESS edwardyoon : Files : /hama/trunk/CHANGES.txt /hama/trunk/bin/hama /hama/trunk/c++ /hama/trunk/c++/pipes /hama/trunk/c++/pipes/.autom4te.cfg /hama/trunk/c++/pipes/Makefile.am /hama/trunk/c++/pipes/Makefile.in /hama/trunk/c++/pipes/aclocal.m4 /hama/trunk/c++/pipes/api /hama/trunk/c++/pipes/api/hama /hama/trunk/c++/pipes/api/hama/Pipes.hh /hama/trunk/c++/pipes/api/hama/TemplateFactory.hh /hama/trunk/c++/pipes/compile /hama/trunk/c++/pipes/config.guess /hama/trunk/c++/pipes/config.sub /hama/trunk/c++/pipes/configure /hama/trunk/c++/pipes/configure.ac /hama/trunk/c++/pipes/debug /hama/trunk/c++/pipes/debug/pipes-default-gdb-commands.txt /hama/trunk/c++/pipes/debug/pipes-default-script /hama/trunk/c++/pipes/depcomp /hama/trunk/c++/pipes/impl /hama/trunk/c++/pipes/impl/HamaPipes.cc /hama/trunk/c++/pipes/impl/config.h.in /hama/trunk/c++/pipes/install-sh /hama/trunk/c++/pipes/ltmain.sh /hama/trunk/c++/pipes/missing /hama/trunk/c++/utils /hama/trunk/c++/utils/.autom4te.cfg /hama/trunk/c++/utils/Makefile.am /hama/trunk/c++/utils/Makefile.in /hama/trunk/c++/utils/aclocal.m4 /hama/trunk/c++/utils/api /hama/trunk/c++/utils/api/hadoop /hama/trunk/c++/utils/api/hadoop/SerialUtils.hh /hama/trunk/c++/utils/api/hadoop/Splitter.hh /hama/trunk/c++/utils/api/hadoop/StringUtils.hh /hama/trunk/c++/utils/config.guess /hama/trunk/c++/utils/config.sub /hama/trunk/c++/utils/configure /hama/trunk/c++/utils/configure.ac /hama/trunk/c++/utils/depcomp /hama/trunk/c++/utils/impl /hama/trunk/c++/utils/impl/SerialUtils.cc /hama/trunk/c++/utils/impl/Splitter.cc /hama/trunk/c++/utils/impl/StringUtils.cc /hama/trunk/c++/utils/impl/config.h.in /hama/trunk/c++/utils/install-sh /hama/trunk/c++/utils/ltmain.sh /hama/trunk/c++/utils/m4 /hama/trunk/c++/utils/m4/hadoop_utils.m4 /hama/trunk/c++/utils/missing /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJob.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJobClient.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPTask.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueLineRecordReader.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/KeyValueTextInputFormat.java /hama/trunk/core/src/main/java/org/apache/hama/bsp/TaskLog.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/Application.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/BinaryProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/DownwardProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplicable.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesApplication.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesBSP.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/PipesPartitioner.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/StreamingProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/Submitter.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/BinaryProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/DownwardProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/MessageType.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/StreamingProtocol.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/protocol/UplinkReader.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/util /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/DistributedCacheUtil.java /hama/trunk/core/src/main/java/org/apache/hama/pipes/util/SequenceFileDumper.java /hama/trunk/pom.xml
          Edward J. Yoon made changes -
          Fix Version/s 0.6.2 [ 12324385 ]
          Fix Version/s 0.7.0 [ 12320349 ]
          Martin Illecker made changes -
          Component/s pipes [ 12321200 ]
          Martin Illecker made changes -
          Labels pipes

            People

            • Assignee:
              Martin Illecker
              Reporter:
              Martin Illecker
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development