Checking for available system memory

Help & Support
1

What I want to do at runtime is check if there is available system memory to run a program.
This is running on a Linux system.
Are there native Crystal resources to do this, or do I need to use native OS commands?
The simplest possible way is desired.

def my_cool_program
   needed_memory =  compute_needed_memory
   return "insufficient memory" unless sufficient_memory?
   # Do stuff with memory
end

def sufficient_memory? (needed_memory)
  available_memory = # determine available system memory
  available_memory > needed_memory
end
2

I know I can run OS commands such as:

$ free 

$ cat /proc/meminfo

but then I have to parse the output to get the available memory info as a numerical value.
I hope Crystal has a simpler and more direct way to do this.

3

Looks like there is GitHub - crystal-community/hardware: Get CPU, Memory and Network informations of the running OS and its processes, but who knows if it still works.

4

I think it would be nice if stdlib could provide some basic system info. Currently, this is not available though.

2 Likes
5

A little search of StackOverflow provided this cli solution.

awk '/Availabe Memory/{print $(NF-3)}' /proc/meminfo
6

This a simpler cli variant.

$ cat /proc/meminfo | grep "MemAvailable"
MemAvailable:   33175076 kB
7

After a little trial and error, this works, and returns the KB of available memory.
You can then convert it to GB, bytes, etc as you need.

def available_KB
  `cat /proc/meminfo | grep "MemAvailable"`.split(' ')[3].to_i
end

You can play around to extract the other /proc/meminfo field values.

I think this should be very straightforward to implement for the std::lib, at least for [Li|U]Nix.

USED IN CODE:

def sufficient_memory? (needed_KB_memory)
  avail_KB = `cat /proc/meminfo | grep "MemAvailable"`.split(' ')[3].to_i
  avail_KB > needed_KB_memory
end
8

After thinking about it, this allows for a bit more universal usage.

def available_KB
  `cat /proc/meminfo | grep "MemAvailable"`.split(' ')[3].to_i
end

Then in code:

return "Insufficient Memory" unless available_KB > needed_KB
9

Someone in the Rust forum (where I asked the same question for doing it in Rust) pointed out I can simplify the cli command. Here’s it with the shortened|tested cli command.

def available_KB
  `grep MemAvailable /proc/meminfo`.split(' ')[3].to_i
end

This is not just shorter, and easier to read and follow, and with better symantics.
It now only uses grep (or equivalent) and /proc/meminfo as Nix dependencies.
Crystal already has a grep so only the info in /proc/meminfo is an external dependency.

10

Here are a couple more SysMem methods.
Put them all in a file on your system and run as follows.

# SysMemMethods.cr

def available_sys_mem
  `grep MemAvailable /proc/meminfo`.split(' ')[3].to_i
end

def total_sys_mem
  `grep MemTotal /proc/meminfo`.split(' ')[7].to_i
end

def free_sys_mem
  `grep MemFree /proc/meminfo`.split(' ')[8].to_i
end

puts "Available System Memory = #{available_sys_mem} kB"
puts "Total System Memory = #{total_sys_mem} kB"
puts "Free System Memory = #{free_sys_mem} kB"
--------------------------------------------------------

$ cyrstal run SysMemMethods.cr
Available System Memory = 30484736 kB
Total System Memory = 40436388 kB
Free System Memory = 21327500 kB

ADDED:
These are more ubiquitous for these values:

def available_sys_mem
  `grep MemAvailable /proc/meminfo`.split(' ')[-2].to_i
end

def total_sys_mem
  `grep MemTotal /proc/meminfo`.split(' ')[-2].to_i
end

def free_sys_mem
  `grep MemFree /proc/meminfo`.split(' ')[-2].to_i
end
11

Any particular reason for shelling out? You could process File.read("/proc/meminfo") directly in Crystal.

12

As a general note, the BSDs prefer using LibC.sysctl, because there is no procfs on macOS and OpenBSD.

1 Like
13

Well, I originally asked if there was a native way to do this and wasn’t provided one.
I knew the information I wanted was available via the shell commands, so that’s what I did.

If you can provide (show) a more native Crystal way to do it that would be nice.

14

Prob could just copy what the one shard I linked earlier does. It isn’t system agnostic since it reads the proc file, but doesn’t shell out to anything.

15

On the Ruby side I use djberg96’s sys-* gems where I need things like this, particularly sys-proctable and sys-memory (ref) . These gems are nice because they handle cross-platform against Windows/Mac/Linux. I could see these being used as inspiration for a new shard.

1 Like
16

Basically here’s the Ruby code that captures everything in /proc/meminfo in a hash.

MEMORY_FILE = '/proc/meminfo'
MEMINFO_REGEX = /(.*)?:\s+?(\d+)/.freeze

def memory
  hash = {} 

  File.foreach(MEMORY_FILE) do |line|
    key, value = MEMINFO_REGEX.match(line.chomp).captures
    hash[key] = value.to_i
  end

  hash
end

I tried to translate that to Crystal (I hardly know regexs) as:

def memory_vals
  mem_vals = Hash(String, Int32).new
  File.each_line("/proc/meminfo") do |line|
    #key, value = /(.*)?:\s+?(\d+)/.match(line.chomp)
    key, value = line.match (/(.*)?:\s+?(\d+)/)
    mem_vals[key] = value.to_i
  end
  mem_vals
end

But keep getting this error message.

In availablemem.cr:47:18

 47 | key, value = line.match (/(.*)?:\s+?(\d+)/)
                   ^-
Error: undefined method '[]' for Nil (compile-time type is (Regex::MatchData | Nil))

Can someone clean this up.

17

String#match returns Regex::MatchData | Nil: method documentation

It’s not a regex issue. You just need to handle the return value of the method correctly.

18

OK, this now works.

def sys_mem_info
  mem_info = Hash(String, Int64).new
  File.each_line("/proc/meminfo") do |line|
    key, value = line.match(/(.*)?:\s+?(\d+)/).not_nil!.to_s.split
    mem_info[key] = value.to_i64
  end
  mem_info
end

pp sys_mem_info
{"MemTotal:" => 40436388,
 "MemFree:" => 21216224,
 "MemAvailable:" => 30907248,
 "Buffers:" => 2133020,
 "Cached:" => 6075404,
 "SwapCached:" => 0,
 "Active:" => 7332312,
 "Inactive:" => 5907564,
 "Active(anon):" => 5591920,
 "Inactive(anon):" => 0,
 "Active(file):" => 1740392,
 "Inactive(file):" => 5907564,
 "Unevictable:" => 88,
 "Mlocked:" => 88,
 "SwapTotal:" => 4193292,
 "SwapFree:" => 4193292,
 "Zswap:" => 0,
 "Zswapped:" => 0,
 "Dirty:" => 6696,
 "Writeback:" => 0,
 "AnonPages:" => 5031392,
 "Mapped:" => 1139696,
 "Shmem:" => 560640,
 "KReclaimable:" => 2568336,
 "Slab:" => 2735624,
 "SReclaimable:" => 2568336,
 "SUnreclaim:" => 167288,
 "KernelStack:" => 31568,
 "PageTables:" => 69804,
 "SecPageTables:" => 0,
 "NFS_Unstable:" => 0,
 "Bounce:" => 0,
 "WritebackTmp:" => 0,
 "CommitLimit:" => 24411484,
 "Committed_AS:" => 14086608,
 "VmallocTotal:" => 34359738367,
 "VmallocUsed:" => 106140,
 "VmallocChunk:" => 0,
 "Percpu:" => 7040,
 "HardwareCorrupted:" => 0,
 "AnonHugePages:" => 0,
 "ShmemHugePages:" => 0,
 "ShmemPmdMapped:" => 0,
 "FileHugePages:" => 0,
 "FilePmdMapped:" => 0,
 "CmaTotal:" => 0,
 "CmaFree:" => 0,
 "HugePages_Total:" => 0,
 "HugePages_Free:" => 0,
 "HugePages_Rsvd:" => 0,
 "HugePages_Surp:" => 0,
 "Hugepagesize:" => 2048,
 "Hugetlb:" => 0,
 "DirectMap4k:" => 3312512,
 "DirectMap2M:" => 15945728,
 "DirectMap1G:" => 23068672}

So now you get everything in /proc/meminfo, and can do (values for my system):

meminfo = sys_mem_info

puts "Total Memory = #{meminfo["MemTotal:"].format} KB" # => 40,436,388 KB

puts "Swap Memory = #{meminfo["SwapTotal:"].format} KB" # => 4,193,292 KB
1 Like
19

Just for completeness, for windows it can done like this:

  lib LibKernel32
    struct MemoryStatusEx
      length : UInt32
      memory_load : UInt32
      total_phys : UInt64
      avail_phys : UInt64
      total_page_file : UInt64
      avail_page_file : UInt64
      total_virtual : UInt64
      avail_virtual : UInt64
      avail_extended_virtual : UInt64
    end

    fun GlobalMemoryStatusEx(lpbuffer : MemoryStatusEx*) : LibC::BOOL
  end

  memory_status_ex = LibKernel32::MemoryStatusEx.new
  memory_status_ex.length = sizeof(LibKernel32::MemoryStatusEx)
  LibKernel32.GlobalMemoryStatusEx(pointerof(memory_status_ex))

  memory_system_available = memory_status_ex.avail_phys
  memory_system_total = memory_status_ex.total_phys