Some of my understandings regarding `ExecutionContext::Parallel`, feedback welcome

Help & Support
1

ExecutionContext::Parallel is probably the most exciting things will enabled by default in incoming release, I write some note use Chinese, following content translate by GPT from Chinese, probably make mistake, please feedback，or clarify the uncertain parts.

especially, There is little documentation available for ExecutionContext::Parallel， I would appreciate it if a core developer, specifically @ysbaddaden, could provide some clarification.

———–

The parallel primitive: ExecutionContext::Parallel

Below, I’ll abbreviate execution context as EC, and use Thread specifically to mean an operating system thread.

An EC creates and manages a dedicated pool of one or more schedulers, and controls how schedulers run on threads (M:N schedulers:threads).

A scheduler is not an execution unit permanently bound to a particular Thread; it may run on different threads over time.

For example:

  • As the number of Fibers increases and the demand for parallelism grows, more schedulers may become active (which in turn starts more system threads).

  • If the number of Fibers drops and the need for parallelism decreases, schedulers may pause (not exit), and the level of parallelism decreases.
    The Thread resources associated with those schedulers may be released.(uncertain)

  • When the number of Fibers increases again, the same scheduler may end up associated with a newly started operating system thread.(uncertain)

    The benefit of this design is that a scheduler does not need to stay permanently bound to a single thread, which reduces the cost of thread management. It also allows the runtime to dynamically increase or stop the number of active schedulers depending on load.

    The relationship between schedulers and Threads is maintained by the Crystal runtime (the EC) itself.

    Fibers : Schedulers : Threads
    F M N

    Fiber::ExecutionContext::Parallel.new(“workers”, 8) # Here M is set to 8

    Here, M is the maximum number of schedulers that may be started (#capacity).
    Initially, only 2 schedulers might be started, corresponding to 2 threads; in that case, N is 2 (#size), and it may grow during execution max to 8 (See the example below.)

    The relationship from M → N changes dynamically depending on the number of Fibers and the current demand for parallelism. In other words, M >= N，the number of Threads can never exceed the number of schedulers.

    Schedulers are responsible for running, suspending, and switching fibers.

    Within the same EC, fibers running on different schedulers can run in parallel.
    Fibers in different execution contexts can also run in parallel.

    A fiber cannot move across ECs, but the same Fiber may run on different Threads during its lifetime.

    There are several ways this can happen:

    1. The Fiber does not move across schedulers, but the scheduler itself is run on a different thread by the EC.

    2. The Fiber moves across schedulers within the same EC. For example, it may suspend on scheduler1 and later resume on scheduler2 (which is running on another thread).

      Work-stealing between schedulers in the same EC is one mechanism that can lead to this.(uncertain, are there other ways to achieve this？)
      For example, when a scheduler runs out of work (its queue is empty or nearly empty), it may “steal” some runnable fibers that have not yet executed from another scheduler in the same EC (uncertain, possibly along with a subtree of child fibers).
      The advantage is that, as long as threads are available, runnable Fibers can be picked up more quickly by idle schedulers, improving load balancing and parallel utilization.

      In short:

      each EC manages a number of schedulers; each scheduler maintains its own queue of runnable fibers and run on a system thread. As a result, multiple schedulers within the same EC can run in parallel.

      Conceptually, this is close to Go’s M:N concurrency model

2

The number of system threads can exceed the number of schedulers; even a concurrent execution context can have multiple threads.

What?!

A scheduler may be detached from a system thread when a fiber is doing blocking syscalls (open, getaddrinfo, …) and resumed on another thread, so there can be 2+ threads even when there’s only 1 scheduler.

It doesn’t break concurrency guarantees: when the blocking syscall returns the fiber is immediately re-enqueued to be resumed by the detached scheduler (so fibers never run in parallel). See #15871.

1 Like
3

Thank you so much for kindly pointing out my mistakes.