Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3753

Add approximateNumEntries() to the StateStore interface for metrics reporting

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.10.1.0
    • streams

    Description

      As a developer building a Kafka Streams application, I'd like to have visibility into what's happening with my state stores. How can I know if a particular store is growing large? How can I know if a particular store is frequently needing to hit disk?

      I'm interested to know if there are existing mechanisms for extracting this information or if other people have thoughts on how we might approach this.

      I can't think of a way to provide metrics generically, so each state store implementation would likely need to handle this separately. Given that the default RocksDBStore will likely be the most-used, it would be a first target for adding metrics.

      I'd be interested in knowing the total number of entries in the store, the total size on disk and in memory, rates of gets and puts, and hit/miss ratio for the MemoryLRUCache. Some of these numbers are likely calculable through the RocksDB API, others may simply not be accessible.

      Would there be value to the wider community in having state stores register metrics?

      Attachments

        Issue Links

          Activity

            People

              jeff.klukas@gmail.com Jeff Klukas
              jeff.klukas@gmail.com Jeff Klukas
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: