Exception inside a spawn block *should* crash the program

Help & Support
1

I have a very simple program. What it does is listen to a Postgres channel, when a notification arrives it does some work and then continues to listen again.

It can happen that a notification is somehow corrupted. Then the program should just crash. I have systemd setup to restart it and it is no big deal.

  spawn do
    begin
      PG.connect_listen(ENV["POSTGRES"], "new_measurements") do |n|
        server_id, file_system_id, measurement_id = n.payload.split(",")

        capacity = pg.scalar("SELECT capacity FROM measurements WHERE id = $1", measurement_id).as(Int32)
        if capacity > 90
          pp "panic!"
        end
      end
    ensure
      pg.close
    end
  end
  sleep

The exception is for instance thrown at the pg.scalar query. But I really want to catch all exceptions with an ensure and then crash/exit the program.
When an exception occurs now the program just hangs. It enters the ensure clause and then hangs on the sleep call? I don’t really know.

The spawn and sleep are necessary otherwise the program will exit immediately without processing notifications.

I’m pretty new to Crystal and I’m not sure what the canonical way of handling this is.

1 Like
2

Yes, an exception only “crashes” the current fiber/coroutine, it does not affect the main fiber that spawned it.

The extra spawn should not be necessary, the program exits without the sleep because connect_listen itself spawns the poll event loop in a new fiber: crystal-pg/connection.cr at master · will/crystal-pg · GitHub

Without testing it myself I suspect something like a close wait channel will work for your usecase:

notification_errors = Channel(Exception).new
pg = PG.connect(...)
PG.connect_listen(...) do |n|
  begin
    capacity = pg.scalar(...)
    raise "bad capacity" if capacity > 90
  rescue e : Exception
    notification_errors.send e
  end
end

# This blocks the main fiber until something is send to the channel
e = notification_errors.receive 

pg.close
abort "Loop broke: #{e.message}"
4 Likes
3

That was the missing piece in my understanding! An exception only crashes the fiber. That makes total sense.
I’ve modified my code according to your suggestion and that works like a charm. Clever using a channel like that.

As a cherry on the pie I learned about abort. Thank you, you’ve made my morning.

2 Likes
4

When it comes to desired behavior as opposed to current behavior, I agree that exceptions should bubble up (by default).

Changing so that exceptions bubble up would require a hierarchical structure of spawned fibers though, as otherwise there will be no guarantee that there is somewhere to bubble to. That hierarchy is basically issue #6468, which is no small task to implement.

That could also do away with having to bother with waiting on a channel, which would be a nice win.

5

Maybe crystal could add an option like ruby’s
Thread.abort_on_exception? Not sure if it would be useful but… typically you want to know when a thread dies unexpectedly? :)

6

You could use a helper like this:

def spawn_with_abort(&block)
  spawn do
    block.call
  rescue exc
    exc.inspect_with_backtrace(STDERR)
    abort
  end
end
7 Likes
7

Possibly related, I believe when a fiber dies it outputs something to stderr. Is there any way to change that behavior?

8

Not globally. I suppose you can override Fiber#run to change that. But I don’t think that’s a good idea because there are fibers not initiated by your application.
Best way to deal with that is similar to what I showed in my previous comment: Rescue the exception in the spawned proc and handle it there.
Or wait for [RFC] Structured Concurrency · Issue #6468 · crystal-lang/crystal · GitHub