Seeing Serdar’s gcry shard to replace the BDW garbage collector with one written entirely in Crystal has got me wondering about other alternatives. I haven’t tried his yet in prod, so I can’t speak about how well it works, but one thing that I’ve run into a lot with Boehm is that once a heap page is allocated, it’s frequently retained for the life of the program. I understand why that happens, but that kind of bloat can be irritating.
For a few years, I’ve been wishing for a compacting GC in Crystal, but there’s a compacting allocator developed at the University of Massachusetts called Mesh that removes the need for the GC to do the compacting. I’ve been hoping for a while could be worked into Boehm, but if the GC can be built around the strengths of Mesh instead of shoehorning Mesh into one designed around a different allocator, that might be better.
There was a session at Strangeloop 2019 about Mesh where the presenter used it with Ruby and Redis and it showed meaningful memory improvements even 7 years ago.
16 KB ceiling. Mesh never meshes objects above 16 KB — they go straight to the global heap. A lot of Crystal’s churn is big buffers (strings, arrays), so if most of the pinning is large objects, this won’t touch it.
Randomization overhead. Random offsets are what make meshing effective, but the paper measured ~10% overhead on regular allocation patterns. A GC allocates far more than a typical malloc workload, so that constant compounds.
OS dependency. The whole trick needs shared file-backed page mappings (memfd_create). Linux and macOS are fine; Windows isn’t there yet.
Workload-shaped payoff. SPEC geomean was −2.4%, but a Redis-style fragmented workload was −39%. So: long-running servers with lots of small allocations benefit; short-lived programs get nothing.
Good point. I’d be curious how many objects tend to be larger than 16KB in my apps, tbh. I have a feeling I’m more of a stickler for avoiding allocations than most.
This is true but not very relevant. It’s like the story about the memory leak on missile-guidance software. If the program has a short lifecycle, it’s not worth it to optimize memory usage.
In my limited understanding, implementing Immix (not RC-Immix or LXR) is simpler or at least less complex than Mesh, with the potential of better overall metrics (allocation speed, throughput and memory usage/overhead).
That is not necessarily true, I certainly notice a difference between a program that takes 1 second and one that takes 2 seconds. Especially for things that I run often (say, running the formatter on save).
Very often a very big share of the execution time is spent in the allocator, even for programs with short runtime. Programs that allocate little tend to be fast.
Nobody’s going out of their way to patch in an alternative GC if their program only runs for up to 1 second with the default. Where is this coming from?
@nadvis 16KB is already a hefty allocation (4 pages). I’m not sure any allocator considers large objects for compaction. Immix, for example, has no design for large objects.
For large objects we can have sized arenas and “punch holes” anywhere inside it. Even in the worst case where there’s only one object allocated at the end of the arena, we can tell the OS that we don’t need all the previous pages.
Small allocations, however, are numerous inside a single page. A single object prevents reclaiming the whole page. This is where compaction or meshing gets interesting: we can move objects to create empty pages.
In Immix compaction moves long lived objects to fill holes at the start of the HEAP, so the start is filled with long lived objects, and only short lived objects live at the end. That’s neat. The issue is that it needs a precise GC.
There are papers on conservative variants of the Immix gc but perhaps those are too complex or bad in some other way? What I wonder though is how it would play with C extensions, but perhaps a nonissue.
Garbage collection is hard. I want to stick my nose into pretty much anything related to Crystal, but honestly, I barely understand it. Still, it would be a little sad to just give up on it, so I hope I can gradually learn at my own pace by watching YouTube and asking AI questions in my free time.
