Loading a dynamic library at runtime with Crystal::FFI

Help & Support
1

Hello,
coming from this GitHub issue:

I’m in the process of adding a Linux equivalent of AHK Windows’ DllCall to my [AHK_X11](https://github.com/phil294/ahk_x11) on Linux project. Because of this, the normal way of leveraging C bindings is not possible.

There once was a DL module for this but [it's been removed](https://github.com/crystal-lang/crystal/pull/8882).

There’s Crystal::FFI and Crystal::Loader from compiler/crystal which includes useful logic such as parsing /etc/ld.so.conf in Loader.default_search_paths to get a list of available libraries. Is there any reason this module isn’t documented or generally part of the standard lib?

From my research, Crystal::Loader emulates ld but not fully, so GNU linker scripts are not parsed. This means that, using libm as an example, while loader = Crystal::Loader.parse(["-lm"]) should work, it doesn’t because libm.so is not a symlink but a text file (at least on my system). So to be able to specify a library version to at least be able to link the library somehow, you need to make library_filename more dynamic. Full code example:

require "compiler/crystal/ffi"
require "compiler/crystal/loader"

class Crystal::Loader
	def self.library_filename(libname : String) : String
		libname = "lib" + libname if ! libname.starts_with?("lib")
		libname += ".so" if ! libname.match /\.so\b/
		libname
	end
end

call_interface = Crystal::FFI::CallInterface.new Crystal::FFI::Type.double, [Crystal::FFI::Type.double]
loader = Crystal::Loader.new(Crystal::Loader.default_search_paths)
loader.load_library("libm.so.6")
function_pointer = loader.find_symbol("cos")
return_value = 0_f64
arg1 = 3.14
arg_pointers = StaticArray[pointerof(arg1).as(Pointer(Void))]
call_interface.call(function_pointer, arg_pointers.to_unsafe, pointerof(return_value).as(Pointer(Void)))
p! return_value

When I don’t load libm.so.6 explicitly but rather the linker script libm.so (or m in the default library_filename, I’m getting

Unhandled exception: cannot find -llibm (/usr/lib/libm.so: cannot open shared object file: No such file or directory) (Crystal::Loader::LoadError)

What is actually happening I think is all possible paths for libm.so being attempted to load, with /usr/lib being the last one of these. None succeeded in being called internally with dlopen() so eventually an error is shown for the last attempt.

Ideally, I’d like to offer my users a way to call something via e.g. libm/cos. This seems to be impossible right now, unless I build my own Loader?

libmath was just an example, eventually all sorts of complex struct handling library calls should be possible.

Thanks! Long live Crystal :blush:

Links in this post are formatted as they are because

An error occurred: Sorry, new users can only put 2 links in a post.

1 Like
2

Crystal::FFI only serves the needs of the interpreter, so it only provides a subset of functionality of libffi. That’s on purpose because we don’t want to maintain any more complexity in the compiler.

A general purpose implementation should be provided as a separate shard. It can of course take the compiler internals for inspiration.

The purpose of Crystal::Loaderis for the interpreter to behave identical to a compiled program, so it must be able to understand linker arguments. That’s why we built a parser for linker arguments which mimicks the native behaviour.

This is a very specific requirement for the interpreter. I doubt that an understanding of native linker’s CLI arguments would be necessary for most use cases of dynamically loading libraries at runtime.
There’ll probably be a more convenient mechanism for what you want to do.