Hadoop Common
  1. Hadoop Common
  2. HADOOP-6637

Benchmark overhead of RPC session establishment

    Details

    • Type: Test Test
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 0.20.3
    • Component/s: benchmarks
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Measure the latency of RPC session establishments through three mechanisms:

      1. simple - no auth
      2. kerberos authentication
      3. delegation token authentication
      1. miniRPCBenchmark.patch
        18 kB
        Konstantin Shvachko
      2. miniRPCBenchmark-20.patch
        7 kB
        Konstantin Shvachko
      3. miniRPCBenchmark.patch
        18 kB
        Konstantin Shvachko
      4. miniRPCBenchmark.patch
        18 kB
        Konstantin Shvachko
      5. miniRPCBenchmark-21.patch
        7 kB
        Konstantin Shvachko
      6. miniRPCBenchmark-20-100.patch
        17 kB
        Konstantin Shvachko

        Activity

        Hide
        Konstantin Shvachko added a comment -

        The patch includes MiniRPCBenchmark, the test, and the change to the script file, which makes the benchmark runnable using

        bin/hadoop org.apache.hadoop.ipc.MiniRPCBenchmark
        

        An excerpt from JavaDoc:
        MiniRPCBenchmark measures time to establish an RPC connection to a secure RPC server. It sequentially establishes connections the specified number of times, and calculates the average time taken to connect. The time to connect includes the server side authentication time.
        The benchmark supports three authentication methods:

        1. simple - no authentication. In order to enter this mode the configuration file core-site.xml should specify hadoop.security.authentication = simple. This is the default mode.
        2. kerberos - kerberos authentication. In order to enter this mode the configuration file core-site.xml should specify hadoop.security.authentication = kerberos and the argument string should provide qualifying keytabFile and userName parameters.
        3. delegation token - authentication using delegation token. In order to enter this mode the benchmark should provide all the mentioned parameters for kerberos authentication plus the useToken argument option.

        Input arguments:

        • numIterations - number of connections to establish
        • keytabFile - keytab file for kerberos authentication
        • userName - principal name for kerberos authentication
        • useToken - should be specified for delegation token authentication
        • logLevel - logging level, see Level
        Show
        Konstantin Shvachko added a comment - The patch includes MiniRPCBenchmark, the test, and the change to the script file, which makes the benchmark runnable using bin/hadoop org.apache.hadoop.ipc.MiniRPCBenchmark An excerpt from JavaDoc: MiniRPCBenchmark measures time to establish an RPC connection to a secure RPC server. It sequentially establishes connections the specified number of times, and calculates the average time taken to connect. The time to connect includes the server side authentication time. The benchmark supports three authentication methods: simple - no authentication. In order to enter this mode the configuration file core-site.xml should specify hadoop.security.authentication = simple . This is the default mode. kerberos - kerberos authentication. In order to enter this mode the configuration file core-site.xml should specify hadoop.security.authentication = kerberos and the argument string should provide qualifying keytabFile and userName parameters. delegation token - authentication using delegation token. In order to enter this mode the benchmark should provide all the mentioned parameters for kerberos authentication plus the useToken argument option. Input arguments: numIterations - number of connections to establish keytabFile - keytab file for kerberos authentication userName - principal name for kerberos authentication useToken - should be specified for delegation token authentication logLevel - logging level, see Level
        Hide
        Todd Lipcon added a comment -

        Hey Konstantin,

        Will you be posting a table of the results here too? Should be very interesting.

        Show
        Todd Lipcon added a comment - Hey Konstantin, Will you be posting a table of the results here too? Should be very interesting.
        Hide
        Suresh Srinivas added a comment -

        Minor comment - rename TestMiniRPCBencmark.java to TestMiniRPCBenchmark.java (missing 'h')

        Show
        Suresh Srinivas added a comment - Minor comment - rename TestMiniRPCBencmark.java to TestMiniRPCBenchmark.java (missing 'h')
        Hide
        Konstantin Shvachko added a comment -

        A similar becnhmark for the 0.20 branch. It only runs in non-authenticated mode, because obviously there is no security there yet. But we will be able to compare how much overhead the security code adds when we don't use it.

        Show
        Konstantin Shvachko added a comment - A similar becnhmark for the 0.20 branch. It only runs in non-authenticated mode, because obviously there is no security there yet. But we will be able to compare how much overhead the security code adds when we don't use it.
        Hide
        Konstantin Shvachko added a comment -

        Fixed the typo, and slightly improved JavaDoc.
        Todd. I am planning to post results, yes.

        Show
        Konstantin Shvachko added a comment - Fixed the typo, and slightly improved JavaDoc. Todd. I am planning to post results, yes.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12439099/miniRPCBenchmark.patch
        against trunk revision 923619.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 10 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12439099/miniRPCBenchmark.patch against trunk revision 923619. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 10 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/36/console This message is automatically generated.
        Hide
        Konstantin Shvachko added a comment -

        I ran the benchmark on three versions of hadoop

        1. 0.20.1, which does not have any security code, and therefore kerberos and delegation token authentications are not applicable there.
        2. 0.20.100, which contains the latest state of security implementation
        3. 0.22.trunk, which does not have all the latest security patches applied at the time of benchmarking (just for the reference)

        The benchmark creates a connection to the RPC server 1000 times. Each time the RPC server authenticates the client using one of the three authentication methods (no authentication, kerberos, delegation token). The result if the average latency of the connection request.

        The table below shows that

        • when security is turned off the the new code still adds 14% overhead.
        • The overhead for kerberos authentication is predictably huge.
        • The delegation token authentication was intended as a fast alternative to kerberos. It is somewhat faster, but not as nearly as the non-secure version. This should definitely be the focus of future optimizations.
        • 0.22 is 1-2% slower compared to 0.20.100. It is expected to catch up with it, when all latest security contributions are ported to the trunk.
        Version No security Kerberos Delegation Tooken
        0.20.1 0.920    
        0.20.100 1.047 (+14%) 44.670 42.615
        0.22 1.597 (+73%) 45.148 43.455
        Show
        Konstantin Shvachko added a comment - I ran the benchmark on three versions of hadoop 0.20.1, which does not have any security code, and therefore kerberos and delegation token authentications are not applicable there. 0.20.100, which contains the latest state of security implementation 0.22.trunk, which does not have all the latest security patches applied at the time of benchmarking (just for the reference) The benchmark creates a connection to the RPC server 1000 times. Each time the RPC server authenticates the client using one of the three authentication methods (no authentication, kerberos, delegation token). The result if the average latency of the connection request. The table below shows that when security is turned off the the new code still adds 14% overhead. The overhead for kerberos authentication is predictably huge. The delegation token authentication was intended as a fast alternative to kerberos. It is somewhat faster, but not as nearly as the non-secure version. This should definitely be the focus of future optimizations. 0.22 is 1-2% slower compared to 0.20.100. It is expected to catch up with it, when all latest security contributions are ported to the trunk. Version No security Kerberos Delegation Tooken 0.20.1 0.920     0.20.100 1.047 (+14%) 44.670 42.615 0.22 1.597 (+73%) 45.148 43.455
        Hide
        Konstantin Shvachko added a comment -

        A minor correction to JavaDoc, and a merge with current trunk.

        Show
        Konstantin Shvachko added a comment - A minor correction to JavaDoc, and a merge with current trunk.
        Hide
        Konstantin Shvachko added a comment -

        Reflecting changes to the security branch and adding a patch for 0.21.

        Show
        Konstantin Shvachko added a comment - Reflecting changes to the security branch and adding a patch for 0.21.
        Hide
        Jitendra Nath Pandey added a comment -

        +1

        Show
        Jitendra Nath Pandey added a comment - +1
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #205 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/205/)
        . Benchmark for establishing RPC session. Contributed by Konstantin Shvachko.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #205 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/205/ ) . Benchmark for establishing RPC session. Contributed by Konstantin Shvachko.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this.
        The patch would not be possible without Jitendra's help on delegation token issues. Thank you Jitendra.

        Show
        Konstantin Shvachko added a comment - I just committed this. The patch would not be possible without Jitendra's help on delegation token issues. Thank you Jitendra.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #282 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/282/)
        . Benchmark for establishing RPC session. Contributed by Konstantin Shvachko.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #282 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/282/ ) . Benchmark for establishing RPC session. Contributed by Konstantin Shvachko.

          People

          • Assignee:
            Konstantin Shvachko
            Reporter:
            Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development