Just pushed fork_join shard, a multi-threaded execution context for recursively generated and nested fiber workloads.
The core idea comes from the work-first model used by Cilk and Java’s ForkJoinPool. Work stays with the worker that created it, while idle workers can take work from busier ones. That means every spawn doesn’t have to pass through a single shared queue, and recursive workloads can spread across the available workers as they grow.
The implementation is not a direct port. Its queueing and scheduling policy combines ideas proven in several runtimes:
- Tokio influenced the bounded worker queues, batch stealing, overflow handling, and limits on concurrent searching.
- Go influenced fair-share global injection, periodic attention to external work, and the wakeup discipline between searching and parked workers.
- Kotlin’s coroutine scheduler influenced the separation of local and external publication, immediate-task locality, and blocking-worker handling.
- OpenJDK’s
ForkJoinPoolinfluenced randomized stealing, contention control, and compensation around blocking work.
The scheduler works with regular Crystal fibers, so there is no new task type to adopt. Channels, timers, I/O, blocking calls, nested workloads, and live pool resizing all continue to work through Crystal’s existing concurrency model.
Feedback and real-world workload results are welcome.