Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9471

Improve data cache metrics

    XMLWordPrintableJSON

    Details

    • Type: Epic
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.4.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
      None
    • Epic Name:
      Improve data cache metrics
    • Epic Color:
      ghx-label-12

      Description

      Currently, the data cache has the following metrics:

      impala-server.io-mgr.remote-data-cache-hit-bytes
      impala-server.io-mgr.remote-data-cache-miss-bytes
      impala-server.io-mgr.remote-data-cache-total-bytes
      impala-server.io-mgr.remote-data-cache-dropped-byte

      There are several questions that these metrics will not answer, especially when we start to consider changes to eviction algorithms. Here are some questions that we may want to be able to answer:

      1. How much memory is being used to track metadata?
      2. What is the distribution of size of entries in the cache?
      3. How many entries are in the cache?
      4. What are the hit/miss counts (as opposed to the hit bytes)?
      5. What is the actual disk usage (as seen by the OS)?

      This is an epic to track adding metrics to answer these questions (and other similar questions).

        Attachments

          Activity

            People

            • Assignee:
              joemcdonnell Joe McDonnell
              Reporter:
              joemcdonnell Joe McDonnell
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: