# Glib main loop and Fibers integration

**URL:** <https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156>\
**Category:** Help & Support\
**Created:** [November 10, 2023, 3:08pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156 "2023-11-10T15:08:14Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [November 10, 2023, 3:08pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/1 "2023-11-10T15:08:15Z")

</div>

Hi, I was playing with GLib main loop to try to let it integrate well with Crystal Fibers, so people could write GTK aplpications using crystal fibers.

So far, so good, I’m basically using the approach that I knew about in some @asterite comment in this forum, i.e. create a thread; call the blocking function; `Fiber.yield`; return.

If I compile my example to a executable it works, however if I try to wrap it in a test case it hangs and never returns unless -Dpreview\_mt.

For the sake of simplicity I removed any GLib binding code and called the C functions directly.

```auto
require "spec"

@[Link("glib-2.0", pkg_config: "glib-2.0")]
lib LibGLib
  fun g_main_context_new : Pointer(Void)
  fun g_main_loop_new(context : Pointer(Void), is_running : LibC::Int) : Pointer(Void)
  fun g_main_loop_run(this : Void*) : Void
  fun g_main_loop_quit(this : Void*) : Void
  fun g_main_loop_is_running(this : Void*) : LibC::Int
end

def main_loop_run(loop : Pointer(Void))
  channel = Channel(Nil).new
  # Need to use thread.new to be sure it will have its own thread and not
  # share it with other fibers.
  Thread.new do
    puts "main loop run on thread #{Fiber.current.name}!"
    LibGLib.g_main_loop_run(loop)
    channel.send(nil)
    puts "main loop thread finished"
    Fiber.yield
  end
  puts "waiting for channel msg from main loop thread"
  channel.receive
  puts "Got the msg!"
end

describe "Fiber integration with GLib main loop" do
  it "works" do
    ctx = LibGLib.g_main_context_new
    loop = LibGLib.g_main_loop_new(ctx, 0)

    spawn(name: "quit") do
      while LibGLib.g_main_loop_is_running(loop).zero?
        puts "waiting main loop to start..."
        Fiber.yield
      end
      puts "calling 'g_main_loop_quit' from thread #{Fiber.current.name}!"
      LibGLib.g_main_loop_quit(loop)
      puts "quit fiber finished!"
    end

    main_loop_run(loop)
    puts "main loop call returned!"
    puts "Why I don't quit!! whyyyy!!??? 😭️"
  end
end

```

Running this with `crystal spec fibers_spec.cr -Dpreview_mt` works fine as expected, but if I remove the `-Dpreview_mt` it hangs after finish the test case:

```plaintext
$ crystal spec fibers_spec.cr 
waiting for channel msg from main loop thread
main loop run on thread main!
calling 'g_main_loop_quit' from thread quit!
quit fiber finished!
main loop thread finished
Got the msg!
main loop call returned!
Why I don't quit!! whyyyy!!??? 😭️
.

```

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![MistressRemilia](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mistressremilia/32/2585_2.png) [@MistressRemilia](https://forum.crystal-lang.org/u/MistressRemilia)\
**Post date:** [November 10, 2023, 10:01pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/2 "2023-11-10T22:01:17Z")

</div>

Last night I ran into a very similar “why can’t this quit?!” issue and found it was because I had Channels with no size. Changing their size to 10 (an arbitrary value) fixed everything. I had forgotten that Channels are not like general purpose message passing queues, and if the Fiber that would normally drain the Channel was already dead, I couldn’t send a new message and things would just hang.

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [November 11, 2023, 10:56am UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/3 "2023-11-11T10:56:18Z")

</div>

The weird part is that if I remove the Spec stuff and compile and run it as a normal app it works, but the code running as a test case hangs.

---

<div class="post-metadata">

**Author:** ![yxhuvud](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/yxhuvud/32/27_2.png) [@yxhuvud](https://forum.crystal-lang.org/u/yxhuvud)\
**Post date:** [November 11, 2023, 3:43pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/4 "2023-11-11T15:43:30Z")

</div>

> [@hugopl](#):
>
> What am I doing wrong?

Don’t mix Threads you have created yourself with anything that involves the event loop. Ie both channels and yielding/sleeping will not behave the way you want as there is no run loop and the newly created thread is simply not integrating with the event loop. As soon as anything goes to sleep it won’t wake up again. If scheduled manually any fibers from those threads would either not come back at all or come back on one of the regular threads. What can be done in manually created threads is very, very limitited and they should probably be considered an internal interface. There has also been recent work in making them more private.

There is at least one [library](https://github.com/yxhuvud/nested_scheduler) that provide threads that have run loops and integrate with the event loops, but that comes from monkey patching all over a lot of private interfaces and it can break (and has!) whenever a new release have been done.

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [November 12, 2023, 4:30pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/5 "2023-11-12T16:30:39Z")

</div>

In the code snippet I create the thread myself because with `spawn`, even with `-Dpreview_mt` isn’t guaranteed that the block will not share the thread with other fibers, in this case that I do a call to a blocking C function any other fiber running in this same thread will never run until the C function returns and I call `Fiber.yield`. So I believe that this is a use case where using `Thread.new` (that was soft removed from stdlib) is not only valid but the only way to make this work.

---

<div class="post-metadata">

**Author:** ![jgaskins](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jgaskins/32/2449_2.png) [@jgaskins](https://forum.crystal-lang.org/u/jgaskins)\
**Post date:** [November 13, 2023, 4:06am UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/6 "2023-11-13T04:06:16Z")

</div>

I’m not 100% sure, but this feels like the same reason that [`will/crystal-pg`](https://github.com/will/crystal-pg) was ported to use Crystal for the wire protocol instead of continuing to use `libpq` under the hood. A C library that blocks the thread blocks the _entire_ thread, including the Crystal fiber scheduler, until its blocking condition is met.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [November 13, 2023, 1:29pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/7 "2023-11-13T13:29:40Z")

</div>

> [@hugopl](#):
>
> The weird part is that if I remove the Spec stuff and compile and run it as a normal app it works

Even in single-thread?

When I was doing [GitHub - bcardiff/crystal-fswatch](https://github.com/bcardiff/crystal-fswatch) fswatch library also needs a dedicated thread. For using this in crystal single-thread mode something that won’t work is using channels to communicate between threads. I think this is something that could be affecting your code.

The runtime in single-thread mode does not allow channels to be used in custom threads. (Unless I missed some updates 🙈)

To workaround that, in the library I created something called [ThreadPortal](https://github.com/bcardiff/crystal-fswatch/blob/master/src/thread_portal.cr). It’s a plain wrapper on channel when MT, but a IO based sync for single thread.

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [November 13, 2023, 1:31pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/8 "2023-11-13T13:31:26Z")

</div>

> [@jgaskins](#):
>
> A C library that blocks the thread blocks the _entire_ thread, including the Crystal fiber scheduler, until its blocking condition is met.

Exactly, this is why I create the thread using `Thread.new`, to be sure the [Crystal::Scheduler](https://github.com/crystal-lang/crystal/blob/1.10.1/src/crystal/scheduler.cr#L25) doesn’t play any role with this thread and let it be happy and isolated.

As I said, it works fine with/without `-Dpreview_mt`, but when I use the `spec` library it hangs, and hangs after all my code finish as show in the debug messages. I need to dig into spec code to find why…

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [November 13, 2023, 6:02pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/9 "2023-11-13T18:02:45Z")

</div>

> [@bcardiff](#):
>
> Even in single-thread?

Yes, because even on single thread `Thread.new` creates a new real thread no matter what.

If I comment the lines 1, 26, 27, 45 and 46 and run the code with/without `-Dpreview_mt` it works. So the issue seems inside the Spec code.

```Text
$ crystal run spec/fibers_spec.cr 
waiting for channel msg from main loop thread
main loop run on thread main!
calling 'g_main_loop_quit' from thread quit!
quit fiber finished!
main loop thread finished
Got the msg!
main loop call returned!
Why I don't quit!! whyyyy!!??? 😭️
hugo ~/src/gi-crystal  fibers  2.7.6p219 14:57:24
$ crystal run -Dpreview_mt spec/fibers_spec.cr 
waiting main loop to start...
waiting main loop to start...
waiting main loop to start...
waiting main loop to start...
waiting main loop to start...
waiting main loop to start...
waiting main loop to start...
waiting for channel msg from main loop thread
main loop run on thread main!

calling 'g_main_loop_quit' from thread quit!
quit fiber finished!
main loop thread finished
Got the msg!
main loop call returned!
Why I don't quit!! whyyyy!!??? 😭️
hugo ~/src/gi-crystal  fibers  2.7.6p219 14:57:33

```

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [April 9, 2024, 10:22pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/10 "2024-04-09T22:22:38Z")

</div>

I revisited this issue today because I need to do some network operations in a GTK application of mine without freeze the API, so I think I found a solution for GLib main loop integration with Crystal main loop without going through the deeps of `Crystal::EventLoop` libevent implementation.

```Crystal
@[Link("glib-2.0", pkg_config: "glib-2.0")]
lib LibGLib
  fun g_main_context_default : Void*
  fun g_main_context_iteration(ctx : Void*, may_block : Int32) : Int32
  fun g_timeout_add_seconds(interval : UInt32, func : Void* -> Int32, data : Void*) : UInt32
end

def glib_counter(_data)
  puts "glib counting"
  1
end

def crystal_counter
  loop do
    sleep(1)
    puts "crystal counting"
  end
end

# Create a GTK main loop context.
ctx = LibGLib.g_main_context_default

# Start GLib counter, triggered by GLib main loop events.
LibGLib.g_timeout_add_seconds(1, ->glib_counter(Void*), Pointer(Void).null)
# Start a Crystal counter, triggered by Crystal main loop events.
spawn crystal_counter

# Wait 10 seconds then quit.
spawn { sleep(10); exit }

Thread.new do
  loop do
    # Let the glib main loop run, blocking the thread if there's no event yet.
    LibGLib.g_main_context_iteration(ctx, 1)
  end
end

Channel(Nil).new.receive

```

This POC works for this plain GLib code, next step is to change this to work with [g\_application\_run](https://gitlab.gnome.org/GNOME/glib/-/blob/main/gio/gapplication.c?ref_type=heads#L2588)

it should print

```plaintext
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting
crystal counting
glib counting

```

---

<div class="post-metadata">

**Author:** ![ysbaddaden](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/ysbaddaden/32/2252_2.png) [@ysbaddaden](https://forum.crystal-lang.org/u/ysbaddaden)\
**Post date:** [April 11, 2024, 12:41pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/11 "2024-04-11T12:41:17Z")

</div>

Nice! It works because threads happen to lazily start their local scheduler + dedicated event loop when you trigger a fiber reschedule or enqueue, which any IO will trigger.

You must enable `-Dpreview_mt`so the stdlib becomes thread safe (Schedulers, IO, Channel, …). You may disable the multiple worker threads with `CRYSTAL_WORKERS=1` if you only need one thread for Crystal + another for GLib / Gtk. I’m doing just that in a tiny Gtk3 app (very small lived, yet capable to run HTTP requests without blocking the UI).

That works but… without any guarantees (it’s hacky, at best). This is something we aim to fix with [RFC 0002: MT Execution Contexts by ysbaddaden · Pull Request #2 · crystal-lang/rfcs · GitHub](https://github.com/crystal-lang/rfcs/pull/2) for example.

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [June 21, 2024, 5:42pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/12 "2024-06-21T17:42:39Z")

</div>

it worked in this small example… but I’m really not sure if this will work for real.

I believe that the integration with the GLib main loop would be a good use case for this new API.

---

<div class="post-metadata">

**Author:** ![ysbaddaden](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/ysbaddaden/32/2252_2.png) [@ysbaddaden](https://forum.crystal-lang.org/u/ysbaddaden)\
**Post date:** [June 23, 2024, 1:07pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/13 "2024-06-23T13:07:32Z")

</div>

@hugopl Yes. The in progress implementation is at [GitHub - ysbaddaden/execution\_context](https://github.com/ysbaddaden/execution_context) and there’s a sample for Gtk+ 3 that uses `ExecutionContext::Isolated`. It works like a charm :)

---

<div class="post-metadata">

**Author:** ![hugopl](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hugopl/32/970_2.png) [@hugopl](https://forum.crystal-lang.org/u/hugopl)\
**Post date:** [June 27, 2024, 11:32pm UTC](https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/14 "2024-06-27T23:32:14Z")

</div>

Nice 💟

GTK4 uses the very same GLib, Gio and GObject libraries, so your in progress implementation must work on GTK4.

I plan to let [rtfm](https://github.com/hugopl/rtfm) be able to download docsets, so I’ll probably test this with your in progress implementation to avoid having to create that `_async` operations in Gio. [tijolo](https://github.com/hugopl/tijolo) also have a bunch of things that must not be in the UI thread and are right now.