I’ve done everything I can think of to understand what’s causing this, and what I’m left to conclude is when you try to spawn too many threads something in the implementation is broken.
In the example below, the input is 15_000_000_000_000, which uses a generator which causes 1485 threads to be spawned. This works correctly.
➜ CRYSTAL_WORKERS=8 ./twinprimes_ssoz 15000000000000
threads = 8
using Prime Generator parameters for P13
segment size = 1572864 resgroups; seg array is [1 x 24576] 64-bits
twinprime candidates = 741758242500; resgroups = 499500500
each of 1485 threads has nextp[2 x 274802] array
setup time = 0.010818 secs
perform twinprimes ssoz sieve
1485 of 1485 twinpairs done
sieve time = 1040.380192 secs
total time = 1040.39101 secs
last segment = 902612 resgroups; segment slices = 318
total twins = 23097407328; last twin = 14999999998692+/-1
But when the input is 16_000_000_000_000, it switches to using a larger generator that spawns off 22,275 threads, which causes the errors to begin after reaching about 179 threads/spawns.
➜ CRYSTAL_WORKERS=8 ./twinprimes_ssoz 16000000000000
threads = 8
using Prime Generator parameters for P17
segment size = 1572864 resgroups; seg array is [1 x 24576] 64-bits
twinprime candidates = 698125408200; resgroups = 31341208
each of 22275 threads has nextp[2 x 283139] array
setup time = 0.032408 secs
perform twinprimes ssoz sieve
175 of 22275 twinpairs doneUnhandled exception in spawn: Arithmetic overflow (OverflowError)
from ???
from ???
from ???
from ???
from ???
from ???
Unhandled exception in spawn: Arithmetic overflow (OverflowError)
from ???
from ???
from ???
from ???
from ???
from ???
Unhandled exception in spawn: Arithmetic overflow (OverflowError)
I’ve verified over and over that the internal math of the spawned code does not overflow. I’ve run the code with larger inputs, but restricted it to the generator that spawns the 1485 threads, and it produces correct outputs.
There is something happening with spawning larger number of threads that’s causing the problems, and I haven’t been able to restrict the process to the offending issue(s).
I think this is something that needs to be understood and fixed before 1.0, because really, 22K threads is not allot in many numerical algorithms, like this.
Here’s the code for just the spawning of the multi-threads.
cnts = Array(UInt64).new(pairscnt, 0)
lastwins = Array(UInt64).new(pairscnt, 0)
done = Channel(Nil).new(pairscnt)
restwins.each_with_index do |r_hi, i|
spawn do
lastwins[i], cnts[i] = twins_sieve(r_hi, kmin, kmax, kb, start_num, end_num, modpg, primes, resinvrs)
print "\r#{i + 1} of #{pairscnt} twinpairs done"
done.send(nil)
end end
pairscnt.times { done.receive } # wait for all threads to finish
Here’s the source code, with compiling and running instructions at the top.
. The way I found out it was that was: