[MESOS-758] Incorrect memory statistics are reported under linux

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0, 0.13.0, 0.14.0, 0.14.1, 0.14.2
• Fix Version/s: 0.15.0
• Assignee: Eric W. Biederman
• Reporter: Eric W. Biederman
• Created: 22/Oct/13 10:20
• Updated: 13/Nov/13 03:35
• Resolved: 13/Nov/13 03:35

Description

Under linux when using the cgroup_isolator only the number of non file backed pages are reported in the memory rss. This excludes all text pages in the binaries and all page cache pages. As a result of the incorrect reporting users are very confused when they hit their memory limits when it appears they are gigabytes away from them.

The correct number to report is from the file memory.usage_in_bytes.

It would also be nice to have:
memory.stat:total_rss – This is the number of bytes that are not backed by files.
memory.stat:total_cached – This is the number of bytes that are backed by files.
memory.stat:total_mapped – This is the number of bytes backed by files that are mmaped into processes.

memory.stat:rss that is currently reported is wrong in two dimensions.

• It does not include child cgroups
• It does not include any file backed pages, so it is strictly impossible to run the job with the limit set to the number that is reported.