Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Recently, we are tuning the performance of our topology and deploying some theoretical performance models that heavily rely on the metrics of query arrival rates. We found a bug in DisruptorQueue that leads to severe underestimation to the queue arrival rates. After further investigation, we finally found that in the current implementation of DisruptorQueue, the arrival rates are actually measured as the number of batches of tuples rather than the actual number of tuples, resulting in significant underestimation of the arrival rates.
To be more specific, in DisruptorQueue.publishDirectSingle() and DisruptorQueue.publishDirect() functions, objects containing tuples are published to the buffer and the metrics are notified by calling _metric.notifyArrivals(1). This works fine when the object is simply a wrapper of a single tuple. However, the object could also be an instance of ArrayList<AddressedTuple> or HashMap<Integer, ArrayList<TaskMessage>>. In such case, we should get the actual number of tuples in the object and notify the _metrics with the right value.
