C binding issue using `out`

Help & Support
1

I am writing a shard which wraps the librdkafka C library. I have this fun defined:

  fun kafka_new = rd_kafka_new(t : Int32, conf : ConfHandle, errstr : UInt8*, errstr_size : LibC::SizeT) : KafkaHandle

Here is the documentation for the rd_kafka_new C function.

I am trying to handle errors when calling this method and getting the error message from the errstr pointer.

@handle = LibRdKafka.kafka_new(LibRdKafka::TYPE_CONSUMER, conf, out errstr, 128)
raise "Kafka: Unable to create new consumer: #{errstr}" unless errstr == LibRdKafka::OK

I am getting the following output:

Program hit a breakpoint and no debugger was attached

I’m unsure why the exception isn’t getting raised. Along with this, I am unsure how to convert the errstr : UInt8* into a Crystal String. Can anyone help me with this?

2

So I appear to have fixed my second point around converting the errstr into a Crystal string:

@handle = LibRdKafka.kafka_new(LibRdKafka::TYPE_CONSUMER, conf, out errstr, 128)
raise "Kafka: Unable to create new consumer: #{String.new(pointerof(errstr))}" unless errstr == Pointer(UInt8).null

But the exception is still not being raised.

3

I think you’re misinterpreting the API.
errstr receives a pointer to previously allocated memory and rd_kafka_new will write to that memory.

The out keyword does not work for this. It just allocates space for a single UInt8 on the stack because the parameter type is UInt8*.
But you need a much larger size and out cannot know that because the size is dynamic, i.e. not encoded in the type.

You can probably do something like this:

error_buffer = uninitialized UInt8[128]
errstr = error_buffer.to_unsafe
@handle = LibRdKafka.kafka_new(LibRdKafka::TYPE_CONSUMER, conf, errstr, error_buffer.bytesize)
error = String.new(errstr)
raise "Kafka: Unable to create new consumer: #{error}" unless error == LibRdKafka::OK

I’m not sure about the error condition, it depends on the type of LibRdKafka::OK.

4

Also, if i am reading things correctly, the docs says that the error condition is if the return value @handle is null. And you are checking the out value instead of the return value

5

Thank you @straight-shoota for the detailed explanation and thanks @bcardiff yes you are right. It works now, here is the full working solution for anyone else struggling with something similar:

error_buffer = uninitialized UInt8[128]
errstr = error_buffer.to_unsafe
@handle = LibRdKafka.kafka_new(LibRdKafka::TYPE_CONSUMER, conf, errstr, error_buffer.size)
raise "Unable to create new consumer: #{String.new(errstr)}" if @handle.null?
6

Total aside, there’s a shard for that, which might be helpful - GitHub - philipp-classen/kafka.cr: crystal-lang wrapper around kafka C library

7

Yeah there’s a few around, but most of them are archived and the API of them isn’t as simple as I think it could be. I like the API of this one but it has no tests and has some small issues - so I’m forking it :slight_smile: