Hadoop Common
  1. Hadoop Common
  2. HADOOP-8070

Add standalone benchmark of protobuf IPC

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.24.0
    • Fix Version/s: 2.0.0-alpha
    • Component/s: benchmarks, ipc
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      To be more comfortable with the switch to protobuf IPC, I'd like to contribute a standalone benchmark which can start any number of client threads and server threads.

      1. hadoop-8070.txt
        19 kB
        Todd Lipcon
      2. hadoop-8070.txt
        19 kB
        Todd Lipcon
      3. hadoop-8070.txt
        16 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          Attached is the benchmark. Sample invocation:

          /usr/lib/jvm/java-6-sun/bin/java -cp /home/todd/git/hadoop-common/hadoop-dist/target/hadoop-0.24.0-SNAPSHOT/share/hadoop/common/lib/*:target/classes:target/test-classes org.apache.hadoop.ipc.PBRPCBenchmark -r 4 -c 30 -s 30 -t 30 -m 1024

          and usage output:

          Protobuf IPC benchmark.
          
          usage: java ... PBRPCBenchmark [options]
          
          Supported options:
           -?,--help                            show this screen
           -c,--clientThreads <numthreads>      number of client threads to run (or 0 to not run client)
           -h,--host <addr>                     host to listen or connect on
           -m,--messageSize <bytes>             size of call parameter in bytes
           -p,--port <port>                     port to listen or connect on
           -r,--serverReaderThreads <threads>   number of server reader threads to run
           -s,--serverThreads <numthreads>      number of server threads (handlers) to run (or 0 to not run
                                                server)
           -t,--time <seconds>                  number of seconds to run clients for
          Must specify at least -c or -s
          Try "--help" option for details.
          
          Show
          Todd Lipcon added a comment - Attached is the benchmark. Sample invocation: /usr/lib/jvm/java-6-sun/bin/java -cp /home/todd/git/hadoop-common/hadoop-dist/target/hadoop-0.24.0-SNAPSHOT/share/hadoop/common/lib/*:target/classes:target/test-classes org.apache.hadoop.ipc.PBRPCBenchmark -r 4 -c 30 -s 30 -t 30 -m 1024 and usage output: Protobuf IPC benchmark. usage: java ... PBRPCBenchmark [options] Supported options: -?,--help show this screen -c,--clientThreads <numthreads> number of client threads to run (or 0 to not run client) -h,--host <addr> host to listen or connect on -m,--messageSize <bytes> size of call parameter in bytes -p,--port <port> port to listen or connect on -r,--serverReaderThreads <threads> number of server reader threads to run -s,--serverThreads <numthreads> number of server threads (handlers) to run (or 0 to not run server) -t,--time <seconds> number of seconds to run clients for Must specify at least -c or -s Try "--help" option for details.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514446/hadoop-8070.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/591//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/591//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514446/hadoop-8070.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/591//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/591//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Improved attachment can also benchmark vs Writable.

          Good news is that, at least for this workload, the protobuf implementation is 50% faster (~30k/sec vs ~20k/sec on my laptop)

          Show
          Todd Lipcon added a comment - Improved attachment can also benchmark vs Writable. Good news is that, at least for this workload, the protobuf implementation is 50% faster (~30k/sec vs ~20k/sec on my laptop)
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514448/hadoop-8070.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/592//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/592//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514448/hadoop-8070.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/592//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/592//console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          Nice job, Todd.

          Show
          Suresh Srinivas added a comment - Nice job, Todd.
          Hide
          Eli Collins added a comment -

          +1 looks great.

          Nit, remove the commented out code below "Set RPC engine to protobuf RPC engine" since you set this via the command line option.

          Show
          Eli Collins added a comment - +1 looks great. Nit, remove the commented out code below "Set RPC engine to protobuf RPC engine" since you set this via the command line option.
          Hide
          Todd Lipcon added a comment -

          Good catch. I actually should be setting the rpc engine above either of the client/server code paths, so that the server-only or client-only cases work. I'll commit this momentarily since it's a trivial change and only in the new benchmark code

          Show
          Todd Lipcon added a comment - Good catch. I actually should be setting the rpc engine above either of the client/server code paths, so that the server-only or client-only cases work. I'll commit this momentarily since it's a trivial change and only in the new benchmark code
          Hide
          Todd Lipcon added a comment -

          Committed to trunk. We can merge this back to 0.23.x when the protobuf stuff is merged there later.

          Show
          Todd Lipcon added a comment - Committed to trunk. We can merge this back to 0.23.x when the protobuf stuff is merged there later.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1800 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1800/)
          HADOOP-8070. Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1800 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1800/ ) HADOOP-8070 . Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1726 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1726/)
          HADOOP-8070. Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1726 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1726/ ) HADOOP-8070 . Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1737 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1737/)
          HADOOP-8070. Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1737 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1737/ ) HADOOP-8070 . Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/)
          HADOOP-8070. Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185)

          Result = FAILURE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/ ) HADOOP-8070 . Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/)
          HADOOP-8070. Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/ ) HADOOP-8070 . Add a standalone benchmark for RPC call performance. Contributed by Todd Lipcon. (Revision 1244185) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244185 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/RPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCCallBenchmark.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MultithreadedTestUtil.java
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have merged this to 0.23.

          Show
          Tsz Wo Nicholas Sze added a comment - I have merged this to 0.23.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development