A not so short post can be found at: https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html
If you want to have a conversation in the twitter land use
5 Likes
OK left a question on that blog. Other question: I assume the GC is compiled with the GC_THREADS flag? https://github.com/ivmai/bdwgc/blob/master/doc/scale.md it would be interesting to see if the other parallel options mentioned on that page affect runtime performance or not…
It might also be interesting to do “uber thread performant” data structures like an array that releases its mutex on a “put” if it’s within the already used space within the array, a hash that has one mutex “per 16th of the range space” or what not to avoid collision (or whatever java’s https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentHashMap.html does).
Then follow java’s lead and have versions of every data structure, one that is “fast” one that is “safe”?
That doesn’t sound exactly fun…the only way around it I can imagine would be to make all structures immutable ex: https://github.com/lucaong/immutable (actually I had the idea the other day that channels could be made to only accept “immutable” or “frozen” data structures but…those might be too violent of a change…).
1 Like
Very cool we are getting new updates to parallelism. Will try it out sometime.
I personally would like a better way to unblock channels, if there is not one already I don’t know about.
I wrote a fuzzer using Crystal that runs worker fibers to generate, send, and receive network messages. All those network messages are pumped into a channel, where the main logic fiber receives them and acts upon the information. However, when quitting the program and shutting down the worker fibers, the main logic fiber gets stuck waiting on something to come through the channel, so I devised a UUID “unblock” message that could be sent to unblock the channel and continue exiting. I would love for a way to do this cleaner, like using a Channel(T)#unblock method to force anything waiting for Channel(T)#receive to continue, but I know this won’t work because then all Channel(T)#receive calls would have to be able to return nil as well.
The better route would probably be a Channel#nonblock_receive, that returns nil or in Ruby there is an IO::EAGAINWait error that gets thrown on blocking IO operations.
Can’t use use a second “exit” channel and then use select to receive from both channels?
Sorry but could you show me an example of this? I couldn’t find an example in the API docs, but it is mentioned. https://crystal-lang.org/api/0.30.1/Channel.html#select(*ops:SelectAction)-class-method
I didn’t know there was a way to receive on two channels at once. Does it unblock when any of the channels gets information?
It should work exactly like (or very similar to) Go’s select. There are no docs because it was all very experimental.