Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9475

Track size of metadata usage for data cache

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.4.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
      None

      Description

      Currently, the memory used to track metadata for the data cache is untracked. When comparing cache eviction algorithms (or just monitoring the cache), the metadata size is a useful metric for comparison.

      The cache implementation contains a kudu MemTracker ( https://github.com/apache/impala/blob/master/be/src/util/cache/cache.cc#L154 ). This MemTracker does not integrate with Impala's MemTracker hierarchy, and it tracks the usage in the cache (not the metadata size). This should be modified to track metadata size and integrate with Impala's MemTracker hierarchy.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              joemcdonnell Joe McDonnell
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: