Saw How much memory do you need to run 1M concurrent tasks? on Hacker News, which is comparing how much RSS usage to enqueue 1 million idle tasks in various programming languages. I just posted a comment with a Crystal implementation and some results. (Search for compumike if you can’t find it…)
Anyone have a better implementation?
As others in there pointed out, the author just generated code from chatGPT, so there’s little explanation or thought put into making it useful. Kinda pointless thread.
darwin-arm64 RSS levels off at ~7 GiB for a trivial worker (~7 KiB/fiber).
-Dpreview_mt and CRYSTAL_WORKERS=$(nproc) vs. CRYSTAL_WORKERS=1 spawning time is much faster in the latter, presumably due to some intra-thread synchronization to support parallelism. Overall, throughput is greater so it is scaling across cores. Didn’t plot linearity of work rate. Haven’t tried it on Intel or AMD linux-x86_64 or linux-armhf.