Avro
  1. Avro
  2. AVRO-279

StatsRpcPlugin -- plugin to keep RPC stats

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: java
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I'm working on an RPC Plugin that, when enabled on the transceiver, keeps track of statistics. It also exposes a servlet for accessing them.

      1. AVRO-279.patch.txt
        18 kB
        Philip Zeyliger
      2. AVRO-279.patch.txt
        18 kB
        Philip Zeyliger
      3. AVRO-279.patch.txt
        17 kB
        Philip Zeyliger
      4. rpc_stats.html
        0.9 kB
        Philip Zeyliger

        Issue Links

          Activity

          Hide
          Philip Zeyliger added a comment -

          Attaching a patch which includes a plugin that records server start/end points for an RPC, and a servlet that displays them.

          One of the tests has a main() method which lets one "demo" the output. Fun to play with. I've attached sample output here. It's very ugly.

          I've also very naively measured the overhead of the stats collection. It's not small:

          Overhead: 28.7800%. RPC/s: 3422.781569 (with) vs 4407.859213 (without). RPC time (ms): 0.292160 vs 0.226868

          That said, stats processing is not getting in the way of sub-microsecond RPCs. And this server is doing absolutely nothing. The input and output schemas are 'null', so there's not even any parsing to do.

          This patch depends on AVRO-271, AVRO-273, and AVRO-275.

          Show
          Philip Zeyliger added a comment - Attaching a patch which includes a plugin that records server start/end points for an RPC, and a servlet that displays them. One of the tests has a main() method which lets one "demo" the output. Fun to play with. I've attached sample output here. It's very ugly. I've also very naively measured the overhead of the stats collection. It's not small: Overhead: 28.7800%. RPC/s: 3422.781569 (with) vs 4407.859213 (without). RPC time (ms): 0.292160 vs 0.226868 That said, stats processing is not getting in the way of sub-microsecond RPCs. And this server is doing absolutely nothing. The input and output schemas are 'null', so there's not even any parsing to do. This patch depends on AVRO-271 , AVRO-273 , and AVRO-275 .
          Hide
          Philip Zeyliger added a comment -

          Attaching a fresh patch which makes a constructor package private to match with AVRO-273 and AVRO-275.

          I'll also sheepishly admit that I forgot an Apache copyright statement on a file, which the new patch fixes.

          Show
          Philip Zeyliger added a comment - Attaching a fresh patch which makes a constructor package private to match with AVRO-273 and AVRO-275 . I'll also sheepishly admit that I forgot an Apache copyright statement on a file, which the new patch fixes.
          Hide
          Philip Zeyliger added a comment -

          Looking through the javadoc revealed that I'd exposed StatsServlet.DEFAULT_SEGMENTER, which I've now made package-private. new patch.

          Show
          Philip Zeyliger added a comment - Looking through the javadoc revealed that I'd exposed StatsServlet.DEFAULT_SEGMENTER, which I've now made package-private. new patch.
          Hide
          Doug Cutting added a comment -

          I just committed this. Thanks, Philip!

          Show
          Doug Cutting added a comment - I just committed this. Thanks, Philip!

            People

            • Assignee:
              Philip Zeyliger
              Reporter:
              Philip Zeyliger
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development