Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5658

Report details of process memory maps via metrics

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.10.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Backend
    • Labels:

      Description

      To help understand the details of process memory consumption, it would be great to have metrics that answered questions like:

      • How many anonymous huge pages are allocated?
      • How many memory maps is the process using?
      • What is the RSS of the process?
      • What is the virtual memory consumption of the process?

      It looks like /proc/<pid>/smaps is the documented interface for this: https://www.kernel.org/doc/Documentation/filesystems/proc.txt

      I'm thinking that maybe we could have the memory maintenance thread periodically scrape /proc/<pid>/smaps and update a set of metrics like:

      • Number of mappings
      • Total virtual address space
      • Total RSS
      • Total bytes of anon huge pages

      We should also report the transparent huge pages config under:
      /sys/kernel/mm/transparent_hugepage/ (or /sys/kernel/mm/redhat_transparent_hugepage if not present).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tarmstrong Tim Armstrong
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: