Giraph
  1. Giraph
  2. GIRAPH-32

Implement benchmarks to evaluate the performance of message passing

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0
    • Component/s: benchmark
    • Labels:
      None

      Description

      Message passing framework plays an important role in Giraph.
      We need some benchmark programs to evaluate the improvement related to message passing method.

      1. GIRAPH-32.patch
        8 kB
        Hyunsik Choi
      2. GIRAPH-32_2.patch
        8 kB
        Hyunsik Choi

        Issue Links

          Activity

          Hide
          Hyunsik Choi added a comment -

          I attach the patch about this issue.

          This patch includes a benchmark class. In this benchmark, for each vertex, the compute function sends a meaningless message into all edges of the vertex. Actually, I intend this benchmark to send messages into random workers. PseudoRandomVertexInputFormat already generates random edges. I employed it.

          This benchmark allows users to set the size of message bytes and the number of sending messages per edge. This is because I think they are basic factors to evaluate the behavior and performance of some message delivery system. Besides, users can adjust the number of edges per vertex rather than adjusting the number of sending messages per. It allows users to make the sending pattern either more spread or more skewed.

          Anyone can review this?

          Show
          Hyunsik Choi added a comment - I attach the patch about this issue. This patch includes a benchmark class. In this benchmark, for each vertex, the compute function sends a meaningless message into all edges of the vertex. Actually, I intend this benchmark to send messages into random workers. PseudoRandomVertexInputFormat already generates random edges. I employed it. This benchmark allows users to set the size of message bytes and the number of sending messages per edge. This is because I think they are basic factors to evaluate the behavior and performance of some message delivery system. Besides, users can adjust the number of edges per vertex rather than adjusting the number of sending messages per. It allows users to make the sending pattern either more spread or more skewed. Anyone can review this?
          Hide
          Avery Ching added a comment -

          +1, only minor comment is the 'for(' -> 'for (' to fit other code conventions. I imagine that this benchmark will evolve over time (i.e. allows Jake's power law distributed input (GIRAPH-26) to be chosen as input instead of the random edges. But looks good to me! Hopefully it will help you guys in your communication testing.

          Show
          Avery Ching added a comment - +1, only minor comment is the 'for(' -> 'for (' to fit other code conventions. I imagine that this benchmark will evolve over time (i.e. allows Jake's power law distributed input ( GIRAPH-26 ) to be chosen as input instead of the random edges. But looks good to me! Hopefully it will help you guys in your communication testing.
          Hide
          Hyunsik Choi added a comment -

          Good idea! According to which InputFormat we use, we could choose the distribution of destination vertices.

          I attach the patch that corrected coding convention.

          Show
          Hyunsik Choi added a comment - Good idea! According to which InputFormat we use, we could choose the distribution of destination vertices. I attach the patch that corrected coding convention.
          Hide
          Hudson added a comment -

          Integrated in Giraph-trunk-Commit #9 (See https://builds.apache.org/job/Giraph-trunk-Commit/9/)
          GIRAPH-32: Implement benchmarks to evaluate the performance of message passing

          hyunsik : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1172605
          Files :

          • /incubator/giraph/trunk/CHANGELOG
          • /incubator/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
          Show
          Hudson added a comment - Integrated in Giraph-trunk-Commit #9 (See https://builds.apache.org/job/Giraph-trunk-Commit/9/ ) GIRAPH-32 : Implement benchmarks to evaluate the performance of message passing hyunsik : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1172605 Files : /incubator/giraph/trunk/CHANGELOG /incubator/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
          Hide
          Hyunsik Choi added a comment -

          Because this issue got +1, I just committed.

          Show
          Hyunsik Choi added a comment - Because this issue got +1, I just committed.

            People

            • Assignee:
              Hyunsik Choi
              Reporter:
              Hyunsik Choi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development