Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3115

Remove locking in metrics package

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • metrics
    • None

    Description

      Currently there is synchronization inside the call
      Sensor.record(value)
      This one lock covers updating all the metrics associated with the Sensor.

      The nice thing about this is that it makes the implementation of the various stats very simple since they can always assume they execute inside a lock.

      Unfortunately the metrics show up as one of the biggest perf hits in the producer when sending small messages.

      It might be possible to optimize this. One approach would be to try to make the stats themselves threadsafe. Obviously introducing locking in the stats themselves would make things worse than they currently are (since the problem is the overhead of the lock not the contention). However it might be possible to cut-and-paste the code for DoubleAdder or AtomicDouble (neither are included in java 7) and do some tricky coding.

      The hard bits would be SampledStat and Histogram.

      Prior to attempting this, it would make sense to remove the synchronization and benchmark the producer to see how much improvement is actually possible.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jkreps Jay Kreps
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: