This is a request to fix, I hope fixable, issue with multi-threading before 1.0. I have a multi-threaded twinprimes sieve that I’ve done now in Nim, Rust, D, and Crystal. The Crystal implementation is the slowest, but that’s not the immediate issue of concern. Each thread does an independent proc…

[image] asterite: Here’s a version without unions though it’s not working faster for me: foo.cr · GitHub FYI, It has same speed as your first version, but now leaks memory like the original, so it’s an all around no win. In general, the more threads you have to use the faster the results, s…

@jzakiya I have some new code. Can you try it? https://gist.github.com/asterite/3207fe5e31c0b944fd57c754f9dfa56f maybe_faster.cr # This Crystal source file is a multiple threaded implementation to perform an # extremely fast Segmented Sieve of Zakiya (SSoZ) to find Twin Primes <= N. # Inpu…

Here are the results for an input of 1 trillion (1_000_000_000_000): Asterite Version 1: 185.1 secs; constant memory use Asterite Version 2: 185.1 secs; increasing memory use Asterite Version 3: 172.5 secs; constant memory use Original Version: 174.5 secs; increasing memory use So the latest…

It’s not guru wizardry. The language is incomplete. The std has no facolities to help writing concurrent or multithreaded code other than the primitives to build those (spawn and channel). Eventually there will be, and then no wizardry will be needed. I think the next main slowness, as you pointed …

Another improvement could be having a pool of arrays so these are reused. I think that should give a big boost, but it’s harder to code.

[image] asterite: And no, there will never be manual memory management in Crystal And what about -Dgc_none? Looks like I still can use it if I don’t need GC. src/gc/none.cr

Sure, you can do it. Then a program that allocates memory can only run for a few seconds before all your computer’s memory is consumed. To be clear, -Dgc_none is a compiler dev tool to try out code without involving the more complex GC. In reality it never makes sense to use it, and I’d like that f…

Hey @asterite , after studying your code, I just replaced the Arrays with Slices in the two methods twins_sieve and nextp_init in my original version, as you did in your code, and here are the results, for input of 1_000_000_000_000. My original using Arrays: 174.5 secs My original using Slices: 16…

Cool! Yeah, I love optimizing things. It strange that the memory keeps goigg up for you. In my case it never goes beyond 20MB. Maybe a difference of OS? No idea.

[image] asterite: And no, there will never be manual memory management in Crystal. There already is, one can call LibC.malloc as easily as any other C function.

1.0 multi-threading memory use issues

Crystal Contrib

jzakiya August 28, 2020, 6:46pm 32

You may want to follow the development of Nim’s ARC.

https://forum.nim-lang.org/t/5734

Timeline for multithreading support

Topic		Replies	Views
Kudos to Crystal! News	12	2333	December 29, 2021
Timeline for multithreading support Crystal Contrib	31	2912	September 24, 2024
Wait-groups fiber pooling in 1.19	10	448	January 21, 2026
How to Parallelize this? Help & Support	6	905	March 25, 2020
Perplexing spawn arithmetic overflow errors Help & Support	23	694	March 24, 2021

1.0 multi-threading memory use issues

Related topics