Using Rust inside a Crystal program

Help & Support
1

It will be nice to write a Crystal program and when there is a need to squeeze more performance, to use call a Rust code from the Crystal program. In addition it will just be a fun way to learn both languages. Can someone share a step by step instructions for doing it?

Google search sent me to this forum but it didn’t give me concrete steps I can follow.

Thanks!

2 Likes
2

I believe it’s the same way you call C from Crystal.

Read this: https://crystal-lang.org/reference/syntax_and_semantics/c_bindings/
And this: https://www.greyblake.com/blog/2017-08-10-exposing-rust-library-to-c/ (might be outdated, I don’t know where are the official Rust instructions for this)

Essentially, Rust can provide an API like C does. Then you link your object from file from Crystal and that’s it.

3 Likes
3

I’m confused. How do you expect to gain more performance from Rust? Rust and Crystal should be roughly on par performance wise. Both compile to a LLVM backend, so even the optimizations are pretty similar. There are some differences in the runtimes, but I’m not aware that makes a big difference. Many benchmarks show Rust and Crystal performing pretty similar. Even if one had a slight edge over the other, it would most likely not justify the overhead for switching to the other.

This could still be a fun endevour, I just don’t think this makes sense when you’re looking for performance improvement.

2 Likes
4

Rust has a more mature and performant multi-threading environment than Crystal currently has, which could be leveraged.

2 Likes
5

Hello,

Perfs are similar.
I use FFI to be able to use some crates in Crystal, it allows to use Rust’s echo-system.

If it helps, here’s a little POC in bulk.

├── Cargo.lock
├── Cargo.toml
├── Makefile
├── shard.yml
├── spec
│   ├── myapp_spec.cr
│   └── spec_helper.cr
├── src
│   ├── lib.rs
│   └── myapp.cr

Makefile

ifeq ($(shell uname),Darwin)
    EXT := dylib
else
    EXT := so
endif

all: target/debug/myrust.$(EXT)
	LIBRARY_PATH=$(PWD)/target/debug:$(LIBRARY_PATH) crystal build src/myapp.cr -o myrust
	LD_LIBRARY_PATH=./target/debug ./myrust

target/debug/myrust.$(EXT): src/lib.rs Cargo.toml
	cargo build
	cd src

clean:
	rm -rf target myrust myrust.dwarf

Cargo.tml

[package]
name = "crystal-rust"
version = "0.1.0"

[lib]
name = "myrust"
crate-type = ["dylib"]

shard.yml

name: myapp
version: 0.1.0

authors:
  - Nicolas Talle

targets:
  myapp:
    main: src/myapp.cr

crystal: 0.32.1

license: MIT

lib.rs

#[no_mangle]
pub extern "C" fn string() -> *const u8 {
  "Hello World\0".as_bytes().as_ptr()
}

myapp.cr

@[Link("myrust")]
lib R
  fun string() : UInt8*
end

module Myapp
  VERSION = "0.1.0"

  msg = String.new(R.string())
  puts "from Rust #{msg}"
end

And to compile:

make

I use FFI (Rust) with the v0.34, but I have tested this POC only with Crystal v0.32.

To go further, callbacks help well. Pay attention to who has the ownership of the variables.

10 Likes
6

I love stuff like this because it’s like “mad scientist” type stuff :laughing: I can’t wait to see what else you come up with this.

6 Likes
7

There is plenty reason to do this. For example, Rust can compile to WASM and Crystal cannot. Moreover. Rust has many blockchain SDKs whereas Crystal none. Yet Crystal can be used for a lot of the logic and interop. Rails has Helix to do this. Doing this with Crystal would be nice.

2 Likes
8

Crystal can generate WASM as it uses LLVM and WASM is one of LLVM targets

2 Likes
9

I thought there were still issues with this because Crystal was missing a few things to be able to do this?

2 Likes
10

This is correct. Without the WASM ABI added to Crystal, there’s no WASM codegen support (plus allllll the other things outlined in the above post)

11

I know that garbage collection was a proposal on WASM a year or two ago; has that been officially included as part of the standard (/ is it available in the LLVM implementation)? It seems like that would be a major blocker (in addition to threads), and even then we’d probably need to write something to fit that standard (or use someone else’s implementation of WASM libgc).

12

@RespiteSage Please see the above linked post. Would be happy to continue discussion about WASM there, but probably not relevant further in this post about using Rust alongside Crystal.

1 Like
13

In our scenario, one reason is to embed a HTTP/3 proxy library made in Rust. Currently, AFAIK there is no HTTP/3 library available in Crystal.

Being able to use Rust inside Crystal allows us to benefit from the bigger library ecosystem, like the HTTP/3 library from the example above.

1 Like
14

Recently, the Ruby bundler allows you to select the Rust extension when generating a Gem from a template.

bundle gem --ext rust foobar

├── Cargo.toml
├── ext
│  └── foobar
│     ├── Cargo.toml
│     ├── extconf.rb
│     └── src
│        └── lib.rs

ext/foobar/Cargo.toml

[package]
name = "foobar"
version = "0.1.0"
edition = "2021"
authors = ["kojix2 <dummy@mail.com>"]
license = "MIT"
publish = false

[lib]
crate-type = ["cdylib"]

[dependencies]
magnus = { version = "0.4" }

ext/foobar/extconf.rb

# frozen_string_literal: true

require "mkmf"
require "rb_sys/mkmf"

create_rust_makefile("foobar/foobar")

lib.rs

use magnus::{define_module, function, prelude::*, Error};

fn hello(subject: String) -> String {
    format!("Hello from Rust, {}!", subject)
}

#[magnus::init]
fn init() -> Result<(), Error> {
    let module = define_module("Foobar")?;
    module.define_singleton_method("hello", function!(hello, 1))?;
    Ok(())
}

In the case of Ruby, it appears that rb-sys generates the primitive bindings and magnus does the type conversion between Ruby and Rust. (I am not familiar with this at all) An example of actual use is polars-ruby.

How can I do this for the Crystal language? I guess it would be something like this.
Create a thin wrapper library on the target Rust library that implements functions with primitive types as arguments that can be called in C. Set #[no_mangle] and pub extern "C".
Generate the shared library in a Makefile. Link objects from Crystal.

1 Like
15

I am creating a Crystal binding for tiktoken-rs using the method suggested by @nico. I have confirmed that it does indeed work.

tiktoken-cr

But the problem is that I need to write a wrapper for tiktoken-rs. The types used in Rust structures and the types used in Rust functions are not compatible with C. So you need to write your own C API before trying to generate a shared object.

tiktoken-c

In Python and Ruby, attributes are added to Rust’s (struct + trait) to automatically convert types and automatically generate bindings. But such approaches are not compatible across languages. It would be nice if there was a common way to easily call Rust from other languages.

1 Like
16

There is crystal bindings to a C quic/HTTP/3 library:

I have not used it, I don’t know what state it’s in, but it looks like it is used by Invidious so I assume it’s decent

1 Like
17

Is there any chance we could do something similar to what magnus is doing, probably using the macro system? I have seen Bindgen consume C++ APIs at build time, which is pretty awesome. Even if not trivial, if that is possible, I feel like wrapping a Crate ought to be less complicated.

3 Likes
18

That would be amazing. I would like something similar to this. There are many rust packages that can be used.