Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3753

Add approximateNumEntries() to the StateStore interface for metrics reporting

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.1.0
    • Component/s: streams
    • Labels:

      Description

      As a developer building a Kafka Streams application, I'd like to have visibility into what's happening with my state stores. How can I know if a particular store is growing large? How can I know if a particular store is frequently needing to hit disk?

      I'm interested to know if there are existing mechanisms for extracting this information or if other people have thoughts on how we might approach this.

      I can't think of a way to provide metrics generically, so each state store implementation would likely need to handle this separately. Given that the default RocksDBStore will likely be the most-used, it would be a first target for adding metrics.

      I'd be interested in knowing the total number of entries in the store, the total size on disk and in memory, rates of gets and puts, and hit/miss ratio for the MemoryLRUCache. Some of these numbers are likely calculable through the RocksDB API, others may simply not be accessible.

      Would there be value to the wider community in having state stores register metrics?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jeff.klukas@gmail.com Jeff Klukas
                Reporter:
                jeff.klukas@gmail.com Jeff Klukas
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: