valgrind reports a memory leak for simple
puts “Hello World”
compiled crystal script.
is there any method for using valgrind reliably with crystal scripts ?
is there any other mem checker with better crystal support ?
does the GC have a bug itself ?
output:
==387674== HEAP SUMMARY:
==387674== in use at exit: 8,192 bytes in 1 blocks
==387674== total heap usage: 15 allocs, 14 frees, 12,336 bytes allocated
==387674==
==387674== LEAK SUMMARY:
==387674== definitely lost: 8,192 bytes in 1 blocks
==387674==
==387674== Conditional jump or move depends on uninitialised value(s)
==387674== at 0x4873D3F: GC_push_all_eager (mark.c:1489)
==387674== by 0x487C4A4: GC_with_callee_saves_pushed (mach_dep.c:422)
==387674== by 0x487C6F4: UnknownInlinedFun (mark_rts.c:878)
==387674== by 0x487C6F4: GC_push_roots.lto_priv.0 (mark_rts.c:951)
==387674== by 0x4869B5E: GC_mark_some (mark.c:374)
==387674== by 0x486A28C: GC_stopped_mark (alloc.c:875)
==387674== by 0x486C62B: GC_try_to_collect_inner.lto_priv.0 (alloc.c:624)
==387674== by 0x487D7EF: UnknownInlinedFun (misc.c:1367)
==387674== by 0x487D7EF: GC_init (misc.c:927)
==387674== by 0x184F22: *GC::init:Nil (boehm.cr:199)
==387674== by 0x1ADDDA: *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 (main.cr:38)
==387674== by 0x15D065: main (main.cr:130)
what you mean with “cleanup after the program exists” ?
for C program “HelloWorld” the cleanup is perfect.
.
it is the GC job to do it properly , why it does not ?
at least in the LEAK SUMMARY, they will be reported as “suppressed” instead:
==400524== LEAK SUMMARY:
==400524== definitely lost: 0 bytes in 0 blocks
==400524== indirectly lost: 0 bytes in 0 blocks
==400524== possibly lost: 0 bytes in 0 blocks
==400524== still reachable: 0 bytes in 0 blocks
==400524== suppressed: 8,192 bytes in 1 blocks
This is not a memory leak. It’s an allocation that lives as long as the program is alive. The OS deallocates all the memory anyway after the program terminates. So there’s no need for an explicit free.
A memory leak would be memory that becomes unused at some point but is never freed (before the program terminates).
This does not seem to happen in the GO language Hello World.
package main
import "fmt"
func main() {
fmt.Println("Hello, world!")
}
==30012== HEAP SUMMARY:
==30012== in use at exit: 0 bytes in 0 blocks
==30012== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==30012==
==30012== All heap blocks were freed -- no leaks are possible
After showing signal.cr to ChatGPT, AI suggested adding the following at_exit block.
altstack = LibC::StackT.new
altstack.ss_sp = LibC.malloc(LibC::SIGSTKSZ)
altstack.ss_size = LibC::SIGSTKSZ
altstack.ss_flags = 0
LibC.sigaltstack(pointerof(altstack), nil)
# Release at end of program
at_exit {
LibC.free(altstack.ss_sp)
}
The crystal env command will tell you where the standard crystal libraries are located.
As instructed by the AI, I rewrote signal.cr and the memory leak - as claimed by Valgrind - no longer occurs.
==30545== HEAP SUMMARY:
==30545== in use at exit: 0 bytes in 0 blocks
==30545== total heap usage: 15 allocs, 15 frees, 12,336 bytes allocated
==30545==
==30545== All heap blocks were freed -- no leaks are possible
Ha, is this simple method really safe? Are there any interesting side effects?
but valgrind checks after termination .
although it is not clear, whether the GC had enought time,
eventually crystal compiler may call the GC at program termination, for a clean up !?
One of the biggest hurdles for using valgrind with Crystal is that the malloc crystal uses is one libgc provides and thus it isn’t recognized as the memory comes from memory that is mmaped into place. Figuring out how to make valgrind recognize that would be very nice to have - not so much for memory leaks for abovementioned reasons, but because other tools that are parts of valgrind would then be usable in a way they currently is not. Tools like Massif or even more DHAT would be super nice to have available.
Potentially, but there is quite a lot of things that are allocated that is then up to the system to clean up. Constants etc - including lookup tables for certain internal functions. There will also be fibers around as there is nothing that enforce that they are done before the program stops.
As for valgrind finding parts of the gc it doesn’t trust - I feel that is a question that is better directed to the libgc developers directly unless we can find some concrete problems stemming from that that apply to crystal. Getting things to run more clean would of course be nice though…
probably not safe because the process might receive signals while running at_exit
why a signal would be your concern, if your app quits anyway.
having a memory issue is more demanding,
did cost me a whole day to figure that it was not my code producing it,
as i simply was not expecting a compiler to be so “careless”.
In Crystal, when a signal is triggered, the program handles it using the alternate stack. This ensures that signal handling works even if the default stack is corrupted.
However, if the alternate stack memory is freed before the program fully exits, there is a small time gap before the program completely stops. If a signal occurs during this gap, the program might try to run on the freed memory.
To solve this issue, in my second post, I switched back to the default stack before freeing the alternate stack.
(Though, I simply followed ChatGPT’s explanation and instructions!)
You cannot easily compare valgrind reports using hello world with different languages.
The result will depend on the type of garbage collector being used (tracing in the case of: java, .NET, Ruby, …, reference counting in the case of Python, Swift, …) and on what kind of setup the entry code of the runtime performs. If the setup code of the runtime directly allocates memory using system calls those will probably not be freed when valgrind does its check. However, memory allocated using system call will be freed by the OS once the process dies.
Basically, for a script in Java for example you can turn off the GC (there is a parameter for that) and your “script” will start and run faster. At the end of the script, when the JVM dies, all memory allocated while the program was running will be automatically freed by the OS (of course in a case like this you have to make sure that the minimum heap size is correct).
If your program is not a script (a server for example), just make sure you execute the main body of your code for a number of iteration before you exit. This will give you a more meaningful report.
If your program is a script … don’t bother.
The exit code of a runtime (Java, Crystal - small runtime, Ruby, …) or the code automatically generated and linked with a program in C or other similar language can elect to explicitly deallocate memory allocated using system calls. This will avoid the small delay waiting for the OS to free the memory. This probably also explains some of the difference you have seen.
This is something that a lot of people don’t understand. On modern processor and OS in C, malloc allocates memory pages using system calls as it runs out of space in the system memory allocated at startup. There is a library that manages the chunks of memory in the system allocated space. When you free this memory in your program the chunk is marked as free but it is not returned to the system. Valgrind’s main function is to check that the chunks marked as “used” on the heap is eventually marked as “available” so that you don’t start requesting more memory from the OS (system call) and run out of memory.
Many compilers have a switch to force the C memory management library to return memory to the system when this is possible but this will work only in some scenarios. This is not the default as this adds more work to be performed when you free some memory. A long running C process performance can degrade because of the way the memory manager works because the heap can become quite fragmented and the time to find available chunks goes up when you do a malloc (or a New in C++).
In sophisticated GC like the one in the JVM the GC periodically defragments the heap. This is why the Java GC sometimes stops the execution. You cannot move a block of memory while it can be used.
Nothing is perfect.