I'm creating shard to call Rust from Crystal

Community
1

I want to create a shard to help with calling Rust libraries from Crystal.
I’ve already successfully integrated crystal with a rust library called tantivy. You can see my repo for examples on how this was done: bendangelo/tantivy.cr: Provides Crystal bindings for Tantivy. The fast full-text search engine library written in Rust. - Codeberg.org

I want to ask the community if this is a worthwhile endeavor. And if so, what Rust libraries would be most benefitial to the community?

Thanks

Edit: One useful create to have in Crystal would be this: fst

1 Like
2

For complex projects like Tantivy, it makes perfect sense to write bindings rather than port it to Crystal, but since fst is a simple library, what benefit is there to using the Rust implementation over a Crystal port?

3

Hey good question. The benefit is the Crystal version will receive updates from the original Rust project. Maybe fst isn’t a good example because it’s kinda done.

I think leverging the existing project can help with the long term.

4

The idea would be from your shard to help generate the C bindings in Rust, and then link them from Crystal?

5

Yes, it will help generate bindings and provide a better interface to send arrays, structs, strings back and fourth through Rust and Crystal. You can see how this is done in the library Tantivy.cr I posted above. Just sending an array or a string requires a lot of boilerplate code.

I want to streamline this work by providing all the boilerplate. But before I do all this work I want to know the interest and what kind of Rust libraries are needed in the Crystal community.

6

In fact, i want to try it several days ago, but, there is no any usage or doc to references, so, i give up it, so, how to know if tantivy.cr is working?

7

Hi, I just updated the readme and fixed a few things. One thing to note is this wrapper is not ready for use but it’s a good starting point. I will find the time to work on it soon.

What is your use case? Do you want this to be integrated into your web server or use it as a standalone search service?

8

I am looking for a static file web search engine, I’m unsure if tantivy meets the need, because i don’t know how it works

9

I would say it’s perfect for that. It’s by far the best search library and it’s very customizable. It can index 10,000 entries a second, probably more. And it has powerful filtering features. I tried to build a search engine for youtube and it was the only thing that could handle that scale.

I will carve out sometime to keep working on this. Because I would like to use this for my own needs in the future.

1 Like
10

We had the need for a little project; our use case was very specific. We did the boilerplate ourselves, and I think a tool to help with that would be great.

11

Ok great. I can start work on something like that.

12

In other projects of mine I use localsearch and just generate the index file using whatever (it’s just a json file)

13

Are you talking about this? GitHub - lucidfrontier45/localsearch: Rust library for local search optimization

14

Looks like I misremembered the name of the JavaScript search engine. Basically, this code, but in crystal