[#HADOOP-2886] Track individual RPC metrics.

Hadoop Common

Track individual RPC metrics.

Created: 22/Feb/08 11:14 PM Updated: 21/May/08 08:05 PM

Return to search

Component/s:

metrics

Affects Version/s:

None

Fix Version/s:

0.17.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
rpc-metrics.patch	2008-02-27 10:35 PM	girish vaitheeswaran	5 kB
rpc-metrics.patch	2008-02-26 08:59 PM	girish vaitheeswaran	8 kB
rpc-metrics.path	2008-02-25 10:36 PM	girish vaitheeswaran	7 kB
rpcmetrics.patch	2008-03-19 05:21 PM	girish vaitheeswaran	3 kB
rpcmetrics.patch	2008-03-13 06:03 PM	girish vaitheeswaran	3 kB
rpcmetrics.patch	2008-03-13 01:12 AM	girish vaitheeswaran	3 kB

Resolution Date:

20/Mar/08 05:06 PM

Description

« Hide

There is currently no mechanism to track performance metrics at the granularity of a specific RPC. So For e.g. if we wanted to capture average latency for the openFile RPC or for the createFile RPC the current infrastructure does not support that.

The implementation involves having a simple HashMap where every new Rpc metric being added would be inserted into the HashMap. Since there is a mechanism to obtain RPC latencies already (without the name of the specific RPC), the identification of what RPC is involved would be done by doing a lookup on the HashMap.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

girish vaitheeswaran added a comment - 25/Feb/08 10:36 PM

Here is a brief description of the change

MetricsTimeVaryingRate.java

+ mr.incrMetric(name + "_avg_time", (int)getPreviousIntervalAverageTime());

Cast needed since we are converting a long field to an integer and the methid incrMetric does not have a method accepting a long field.

RPC.java

This file deals with dynamically obtaining the rpc latency based on the name of the rpc.

RpcMetrics.java

This file deals with generating the hashmap for the different RPC's.

[ Permlink | « Hide ]

girish vaitheeswaran added a comment - 29/Feb/08 12:16 AM

No tests added since this is a minor enhanacement localized to the rpc metrics code path.

[ Permlink | « Hide ]

girish vaitheeswaran added a comment - 13/Mar/08 01:12 AM

New patch that takes into account the ability to dynamically add new metrics as new rpc's come on board. No extra code is needed.

[ Permlink | « Hide ]

Sanjay Radia added a comment - 13/Mar/08 04:59 PM

Looks good.
2 comments.
1. Hadoop brace style is std java brace style
if ( ) { <- brace is here, not on next line xxxx }
synchronized ( xxx) {

}

2. Since there are some static metrics such as RpcQueueTime there is a potential of
conflicts with rpc methods of the same name.
Hence I suggest that you append the method metrics by "OP_" or something like that.
(Which has the advantage that it obvious that the metrics is an rpc operation/method)

Alternatively you could prefix the static ones by a "_" or something that would not
conflic with method names (since java does not allow methods to start with that character).
This would be cheaper in cost but would break compatibility with the metrics xml file.