Passing Crystal callbacks to C structs with safe user interfaces

Learning Resources
1

Hi, Crystal community!

I’ve been working closely with some C libraries lately and had to solve some problems that I’ve never run into before, dealing with passing callbacks to C structs expecting function pointers. I found the experience fun and interesting, so I wrote up an article talking about it:

https://medium.com/@z64/passing-crystal-callbacks-to-c-structs-with-safe-user-interfaces-6e881aa11ee3

If you give it a read, would love to hear your thoughts or questions here, or on Medium :slight_smile:

3 Likes
2

That is a very useful article indeed. Thanks, Zac!

3

Nice article!

Given that this is a really nice tutorial and it could serve others, can I ask you to make a few changes?

  1. You use Void* for the function pointers. Please no! You are removing all the type safety and benefits that Crystal gives you. With that you can pass any function pointer, even with a wrong signature, and it will break. With correct types you can’t make mistakes, and Crystal will even tell you if you are passing a closure at runtime and give a proper exception instead of a segfault.

Try out this code:

@[Link(ldflags: "#{__DIR__}/test.o")]
lib LibTest
  struct Operations
    work : LibC::Int*, LibC::Int, LibC::Int ->
    log : LibC::Int ->
  end

  fun call(ops : Operations*, a : LibC::Int, b : LibC::Int)
end

# In Crystal, we can represent our callbacks as strongly typed Procs:
add = ->(result : Int32*, a : Int32, b : Int32) do
  result.value = a + b
end

log = ->(result : Int32) do
  puts "result: #{result}"
end

# We make an instance of our Operations struct, assigning a pointer (and *not*
# the Proc itself) as member values respectively:
ops = LibTest::Operations.new(
  work: add,
  log: log
)

# Run call, passing a reference to our ops struct and some values to test:
LibTest.call(pointerof(ops), 1, 2)
LibTest.call(pointerof(ops), 7, 10)

That works fine and it’s type-safe. No closures here, everything’s fine. Now change it to this:

@[Link(ldflags: "#{__DIR__}/test.o")]
lib LibTest
  struct Operations
    work : LibC::Int*, LibC::Int, LibC::Int ->
    log : LibC::Int ->
  end

  fun call(ops : Operations*, a : LibC::Int, b : LibC::Int)
end

# In Crystal, we can represent our callbacks as strongly typed Procs:
closure_data = 10
add = ->(result : Int32*, a : Int32, b : Int32) do
  result.value = a + closure_data
end

log = ->(result : Int32) do
  puts "result: #{result}"
end

# We make an instance of our Operations struct, assigning a pointer (and *not*
# the Proc itself) as member values respectively:
ops = LibTest::Operations.new(
  work: add,
  log: log
)

# Run call, passing a reference to our ops struct and some values to test:
LibTest.call(pointerof(ops), 1, 2)
LibTest.call(pointerof(ops), 7, 10)

When you run it you’ll get an exception with this message:

Unhandled exception: passing a closure to C is not allowed (Exception)

No need to get a segmentation fault and try to understand what went wrong.

And if you write it like this:

ops = LibTest::Operations.new(
  work: ->(result : Int32*, a : Int32, b : Int32) do
    result.value = a + closure_data
  end,
  log: log
)

you get a compile-time error saying “can’t set closure as C struct member (closured vars: closure_data)” because the compiler can detect that you are passing a closure.

Then your example with a nice wrapper and passing a block:

@[Link(ldflags: "#{__DIR__}/test.o")]
lib LibTest
  struct Operations
    work : LibC::Int*, LibC::Int, LibC::Int ->
    log : LibC::Int ->
  end

  fun call(ops : Operations*, a : LibC::Int, b : LibC::Int)
end

class Test
  def initialize
    @ops = LibTest::Operations.new
  end

  def work(&block : Int32, Int32 -> Int32)
    @ops.work = ->(result : Int32*, a : Int32, b : Int32) do
      result.value = block.call(a, b)
    end
  end

  def log(&block : Int32 ->)
    @ops.log = block
  end

  def call(a, b)
    LibTest.call(pointerof(@ops), a, b)
  end
end

test = Test.new

test.work do |a, b|
  a + b
end

test.log do |r|
  puts "value: #{r}"
end

test.call(1, 2)
test.call(7, 10)

Again, you get a compile-time error:

Error in foo.cr:33: instantiating 'Test#work()'

test.work do |a, b|
     ^~~~

in foo.cr:17: can't set closure as C struct member (closured vars: block)

    @ops.work = ->(result : Int32*, a : Int32, b : Int32) do
                                               ^

Good bye segmentation fault, and clear reason :slight_smile:

And there’s no need for you to manually check if something is a closure and raise, the compiler does it for you.

Finally, another alternative to being able to capture multiple callbacks, one per Operation, is to do something like this:

@[Link(ldflags: "#{__DIR__}/test.o")]
lib LibTest
  struct Operations
    work : LibC::Int*, LibC::Int, LibC::Int ->
    log : LibC::Int ->
  end

  fun call(ops : Operations*, a : LibC::Int, b : LibC::Int)
end

class Test
  @@work : (Int32*, Int32, Int32 -> Nil) = ->(result : Int32*, a : Int32, b : Int32) do
    if (current = @@current) && (work = current.work)
      result.value = work.call(a, b)
    end
    nil
  end

  @@log : (Int32 -> Nil) = ->(r : Int32) do
    @@current.try &.log.try &.call(r)
    nil
  end

  getter work, log

  def initialize
    @ops = LibTest::Operations.new
    @ops.work = @@work
    @ops.log = @@log
  end

  def work(&@work : Int32, Int32 -> Int32)
  end

  def log(&@log : Int32 ->)
  end

  def call(a, b)
    Test.set_current(self) do
      LibTest.call(pointerof(@ops), a, b)
    end
  end

  def self.set_current(current : Test)
    old = @@current
    @@current = current
    yield
    @@current = old
  end
end

test = Test.new
test.work do |a, b|
  a + b
end
test.log do |r|
  puts "value: #{r}"
end

test.call(1, 2)
test.call(7, 10)

I’m not sure that will work for multiple threads, or maybe we must know the current Fiber or something before invoking the callback, but it’s just a starting point.

4 Likes