Add real memory usage metrics from smaps

[ClusterM]

At the moment, the memory metric is based only on virtual memory (VmSize), which is misleading in most cases. Virtual memory includes all address space mappings of the process — anonymous allocations, shared libraries, file-backed mappings, and even regions that are never actually loaded into physical RAM. As a result, VmSize often reports extremely large values that do not reflect the real memory footprint of a process. For example, my torrent client shows more than 8 GB of virtual memory usage, while in reality the system as a whole only uses around 2 GB of RAM. This makes VmSize unsuitable for monitoring or comparing services, since it exaggerates memory usage and hides the actual resources consumed.

This commit introduces additional memory statistics based on /proc//smaps_rollup:

• memory_real_pss: proportional set size (shared pages divided among processes), representing the actual memory footprint of the service.
• memory_real: conservative estimate of memory that will be freed if the process exits, based on Anonymous + SwapPss, excluding file-backed shared pages.

These new metrics provide a more accurate view of real memory usage compared to the traditional RSS and virtual memory fields.

smaps-based metrics are disabled by default and can be enabled with the --smaps option.

[miaucl]

@ClusterM Couldn't this be done with psutil?

import psutil
p = psutil.Process()
print(p.memory_info())   # rss, vms, shared, text, data, dirty (from: `/proc/self/statm`)
print(p.memory_full_info())  # includes PSS (if available) (from: `/proc/self/smaps`)

Might be worth a rewrite and not use top anymore but instead /proc/self/statm which is world-readable.

/proc/self/smaps requires root access for the process. This might be a problem in some cases.

[ClusterM]

@ClusterM Couldn't this be done with psutil?

import psutil
p = psutil.Process()
print(p.memory_info())   # rss, vms, shared, text, data, dirty (from: `/proc/self/statm`)
print(p.memory_full_info())  # includes PSS (if available) (from: `/proc/self/smaps`)

Might be worth a rewrite and not use top anymore but instead /proc/self/statm which is world-readable.

/proc/self/smaps requires root access for the process. This might be a problem in some cases.

Yes, psutil can be used. But for PSS you still need access to /proc/<pid>/smaps, which normally requires root or the same uid — there’s no other way around it. On newer kernels /proc/<pid>/smaps_rollup is available and much lighter to parse than the full smaps, but it has the same permission requirement. Without privileges you’re limited to RSS/VMS from /proc/<pid>/statm.

I’ll take a look in my spare time at how the code could be rewritten to use psutil - I think that would indeed be a better approach.

[ClusterM]

Also, psutil does not expose fields like SwapPss, or the detailed PSS breakdown (Pss_Anon, Pss_File, Pss_Shmem, Pss_Dirty), which we need to accurately calculate the real footprint and estimate how much memory will actually be freed if the process exits.

[miaucl]

Also, psutil does not expose fields like SwapPss, or the detailed PSS breakdown (Pss_Anon, Pss_File, Pss_Shmem, Pss_Dirty), which we need to accurately calculate the real footprint and estimate how much memory will actually be freed if the process exits.

Hey, thanks for the insight. It might be even interesting to use psutil for RSS/VMS per default and then optionally extend with root access to /proc/self/smaps OR /proc/self/smaps_rollup as you already proposed, depending on the needs (or kernel version).