1.0 multi-threading memory use issues

Hey @asterite, after studying your code, I just replaced the Arrays with Slices in the two methods twins_sieve and nextp_init in my original version, as you did in your code, and here are the results, for input of 1_000_000_000_000.

My original using Arrays: 174.5 secs
My original using Slices: 161.1 secs

The memory still grows as before, but that’s seem to be due to, as your stated, the incomplete multi-threading model.

I want to say I really appreciate how you jumped on this issue and made the code much better. I have no doubt Crystal will catch up to Nim, Rust, et al, in multi-processing very soon.
Please feel free to use this application as you please as a test case.
Here is the updated gist of my code using Slices.

4 Likes

Cool!

Yeah, I love optimizing things.

It strange that the memory keeps goigg up for you. In my case it never goes beyond 20MB. Maybe a difference of OS? No idea.

There already is, one can call LibC.malloc as easily as any other C function.

My laptop for these results is a 2011 Lenovo, I5 2C|4T, 2.9 GHz, Linux kernel 5.7.15.
Crystal 0.35.1

Actually, I want to experiment with that flag as part of my old idea to use ARC (automatic Reference Counting) as possible option to GC.

2 Likes

And how do you use in crystal? I’ve got no idea and I thought it would be impossible due to the strong type system

Yes, to be absolutely clear, doing so is inherently unsafe and can easily lead to memory leaks! LibC.malloc returns a raw Pointer, which you can then wrap in a slice and pass around like any other, but doing so discards guarantees the compiler offers (memory safety).

That’s where ARC should help.

1 Like

ARC would be a dream come true, for sure! I just don’t feel capable of implementing it personally.

1 Like

I am planning to do it but don’t have enough time. It may be equal or even may be more work than debugger support (I spent 6 months on it).
But I am still willing to work on it as it is an interesting idea to implement.

3 Likes

You mean ARC like the ones implemented in Swift & ObjectiveC with the LLVM support? ( retain and release). That would be awesome, I think.

1 Like

You may want to follow the development of Nim’s ARC.

https://forum.nim-lang.org/t/5734

2 Likes

Yes, exactly that approach.

1 Like