C bindings with callback

Help & Support
1

I’m trying to write some Crystal bindings for the libical C-library.

I found Callbacks - Crystal and tried to use it to adapt the icalparser_get_line function.

Here’s the relevant part of libical.cr:

@[Link("ical")]
lib LibIcal
  # ...
  fun get_line = "icalparser_get_line"(
    parser : LibIcal::IcalParser*, 
    line_gen_func : (LibC::Char*, LibC::SizeT, Void* -> LibC::Char*), 
    data : Void*
  ) : LibC::Char*
end

I then tried to use the function like so:

require "../../src/app.cr"

module Ical
  # The callback for the user doesn't have a Void*
  @@box = Pointer(Void).null

  def self.line_gen_callback(parser : LibIcal::IcalParser, &callback : ((LibC::Char*, LibC::SizeT, Void*) -> LibC::Char*))
    # Since Proc is a {Void*, Void*}, we can't turn that into a Void*, so we
    # "box" it: we allocate memory and store the Proc there
    boxed_data = Box.box(callback)

    # We must save this in Crystal-land so the GC doesn't collect it (*)
    @@box = boxed_data

    # We pass a callback that doesn't form a closure, and pass the boxed_data as
    # the callback data
    LibIcal.get_line(pointerof(parser), -> (s : LibC::Char*, size : LibC::SizeT, data : Void*) {
      p "here"
      p s
      p size
      p data
      # Now we turn data back into the Proc, using Box.unbox
      data_as_callback = Box(typeof(callback)).unbox(data)

      # And finally invoke the user's callback
      data_as_callback.call(s, size, data)
    }, boxed_data)
  end
end

parser = LibIcal.new_parser

File.open("basic.ics") do |file|
  LibIcal.set_gen_data(pointerof(parser), pointerof(file))

  line = -1

  until line == 0
    Ical.line_gen_callback(parser) do |str, size, d|
      p "there"
      line_string = file.gets
      p line_string
      if line_string.nil?
        Pointer(LibC::Char).null
      else
        Intrinsics.memcpy(str, line_string, line_string.size, false)
        str
      end
    end
  end
end

I’m getting the following error:

Invalid memory access (signal 11) at address 0x0
[0x102cd15c0] *Exception::CallStack::print_backtrace:Nil +104 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x102cb7b48] ~procProc(Int32, Pointer(LibC::SiginfoT), Pointer(Void), Nil)@/opt/homebrew/Cellar/crystal/1.11.2/share/crystal/src/crystal/system/unix/signal.cr:131 +320 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x188c83584] _sigtramp +56 in /usr/lib/system/libsystem_platform.dylib
[0x102cbcfd8] ~procProc(Pointer(UInt8), UInt64, Pointer(Void), Pointer(UInt8))@tmp/console/1711652312560_console.cr:17 +152 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x10355e728] icalparser_get_line +216 in /opt/homebrew/Cellar/libical/3.0.17_1/lib/libical.3.0.17.dylib
[0x102e7aea8] *Ical::line_gen_callback<struct.LibIcal::IcalParser, &Proc(Pointer(UInt8), UInt64, Pointer(Void), Pointer(UInt8))>:Pointer(UInt8) +2932 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x102c9e51c] __crystal_main +8888 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x102e7c23c] *Crystal::main_user_code<Int32, Pointer(Pointer(UInt8))>:Nil +12 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x102e7c1a4] *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 +60 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
[0x102ca7a8c] main +12 in /Users/nerdinand/.cache/crystal/crystal-run-eval.tmp
"here"
Pointer(UInt8)@0x16d166490
72057594037927936
Pointer(Void)@0x1031edf60

Any help with this is highly appreciated! I’m pretty new to Crystal, coming from Ruby and Python.

2

My first hunch would be that you try to copy more bytes into str than it accepts, i.e line_string.size > size (btw. correct would be line_string.bytesize). Dunno if that’s the cause for this particular error, but it could certainly cause an invalid memory access.

IO#gets takes a limit parameter which you could set to size to restrict how much bytes it reads.

3

Thanks for the input! Correcting this however doesn’t fix the problem. Seeing as "there" isn’t in the output, it seems the error happens while calling the icalparser_get_line C-function, before the callback-block is even invoked…

5

This seems more complicated than necessary. There is no need to box if the lifetime of the object won’t be longer than the scope it is defined in. So something like

def self.line_gen_callback(parser : LibIcal::IcalParser, &callback : ((LibC::Char*, LibC::SizeT, Void*) -> LibC::Char*))
    LibIcal.get_line(pointerof(parser), callback)
  end

should work in this simple case.

Assuming everything is as claimed. But that is not the case, as the claim that the callback that is passed in doesn’t have a closure. But that is false - the callback refers to file which comes from outside the scope. One of those callbacks probably get the file back as the data argument. Instead of using file directly, use that pointer to construct the file variable.

That said, I usually don’t put functions in the data slot, but rather pointers to instances of objects, that tend to be a lot more flexible.

6

So taking a look at the docs you shared, icalparser_get_line has the following signature in C:

typedef char *(*icalparser_line_gen_func) (char *s, size_t size, void *d);

char* icalparser_get_line(icalparser* parser,
                          icalparser_line_gen_func line_gen_func)

I’m thinking a correct binding would be something like:

# char *s = first LibC::Char*
# size_t size = LibC::SizeT
# void *d = Void*
# return type is LibC::Char*
alias ParserLineGenFunc = Proc(LibC::Char*, LibC::SizeT, Void*, LibC::Char*)

fun get_line = icalparser_get_line(parser : LibIcal::ICalParser*,
                                   line_gen_func : ParserLineGenFunc)

parser = LibIcal.new_parser

File.open("basic.ics") do |file|
  LibIcal.set_gen_data(pointerof(parser), pointerof(file))
  # [snip]
end

I’m curious, how have you defined LibIcal#new_parser and LibIcal::ICalParser? In C, the icalparser_new() function returns icalparser*, so there shouldn’t be any need to use pointerof(parser).

Also, keep in mind the pointerof(file), this is your “data”.

    LibIcal.get_line(pointerof(parser), -> (s : LibC::Char*, size : LibC::SizeT, data : Void*) {
      p "here"
      p s
      p size
      p data
      # Now we turn data back into the Proc, using Box.unbox
      data_as_callback = Box(typeof(callback)).unbox(data)

      # And finally invoke the user's callback
      data_as_callback.call(s, size, data)
    }, boxed_data)

Again, probably need to check the pointerof(parser).

As @yxhuvud mentioned, there isn’t any need to box anything here, nor is there any need for the second callback. The data is the “data” you set earlier with LibIcal#set_gen_data, in other words, it’s your file. LibIcal.get_line(parser, callback) should work.

  until line == 0
    Ical.line_gen_callback(parser) do |str, size, d|
      p "there"
      line_string = file.gets
      p line_string
      if line_string.nil?
        Pointer(LibC::Char).null
      else
        Intrinsics.memcpy(str, line_string, line_string.size, false)
        str
      end
    end
  end

Knowing that data is a File, you can convert it back in to something usable by combining Object#as(type) and Pointer(T). It seems that you’ve fixed the other issues regarding line_string.size and size. I’d also suggest taking a look at your until line == 0 loop condition as line never changes and there doesn’t seem to way out of the loop.

Let me know if you get stuck.