Uploaded image for project: 'Hama'
  1. Hama
  2. HAMA-839

Support NullWritable in Hama Pipes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.6.3
    • 0.7.0
    • bsp core, pipes

    Description

      Support NullWritable in Hama Pipes

      It should be possible to read, write and send NullWritables in Hama Pipes.

      Therefore the void type is supported in C++ templates.
      e.g., PiEstimator (InputKeyClass, InputValueClass and OutputKeyClass are NullWritables)

      -class PiEstimatorBSP: public BSP<string,string,string,double,int> {
      +class PiEstimatorBSP: public BSP<void,void,void,double,int> {
      ...
      -  void cleanup(BSPContext<string,string,string,double,int>& context) {
      +  void cleanup(BSPContext<void,void,void,double,int>& context) {
      ...
           double pi = 4.0 * total_hits / (msg_count * iterations_);
      -    context.write("Estimated value of PI", pi);
      +    context.write(pi);
           }
        }
      

      This is affecting the following C++ methods:

      • Input
        bool readNext(K1& key, V1& value)
        method is available at compile time if BSP<K1!=void, V1!=void, ?, ?, ?>
        bool readNext(K1& key)
        method is available at compile time if BSP<K1!=void, V1==void, ?, ?, ?>
        bool readNext(V1& value)
        method is available at compile time if BSP<K1==void, V1!=void, ?, ?, ?>
          method will not be available at compile time if BSP<K1==void, V1==void, ?, ?, ?>
      • Output
        void write(const K2& key, const V2& value)
        method is available at compile time if BSP<?, ?, K2!=void, V2!=void, ?>
        void write(const K2& key)
        method is available at compile time if BSP<?, ?, K2!=void, V2==void, ?>
        void write(const V2& value)
        method is available at compile time if BSP<?, ?, K2==void, V2!=void, ?>
          method will not be available at compile time if BSP<?, ?, K2==void, V2==void, ?>
      • Message
        void sendMessage(const string& peerName, const M& msg)
        method will not be available at compile time if BSP<?, ?, ?, ?, M==void>
        M getCurrentMessage()
        method will not be available at compile time if BSP<?, ?, ?, ?, M==void>

      Hama Pipes supports NullWritables in SequenceFileInputFormat and SequenceFileOutputFormat.

      The same is valid for the SequenceFile methods:

      • SequenceFile.Reader
        bool sequenceFileReadNext<K, V>(int fileID, K& key, V& value)
        method is available at compile time if <K!=void, V!=void>
        bool sequenceFileReadNext<K>(int fileID, K& key)
        method is available at compile time if <K!=void>
        bool sequenceFileReadNext<V>(int fileID, V& value)
        method is available at compile time if <V!=void>
      • SequenceFile.Writer
        bool sequenceFileAppend<K, V>(int fileID, const K& key, const V& value)
        method is available at compile time if <K!=void, V!=void>
        bool sequenceFileAppend<K>(int fileID, K& key)
        method is available at compile time if <K!=void>
        bool sequenceFileAppend<V>(int fileID, V& value)
        method is available at compile time if <V!=void>

      Attachments

        1. HAMA-839_1.patch
          96 kB
          Martin Illecker
        2. HAMA-839_2.patch
          119 kB
          Martin Illecker

        Activity

          People

            bafu Martin Illecker
            bafu Martin Illecker
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: