Will it be possible to embed the upcoming Crystal interpreter?

Hi all,

I’ve been quite hyped by @asterite 's crystal i presentation ! :)
Currently I don’t have any Crystal code in production yet, however I was concerned by the development iterations slowed down by compile times and also by the lack of convenient debugging facilities that I’m currently used to with dynamic languages (Python in my case).

This Crystal interpreter seems to be an excellent solution in order to tackle those concerns, especially if it runs more or less at the same speed than ruby or python, but I have the feeling it could serve more purposes so I’d like to get the core’s team opinion if such a thing would be possible.

In some projects, embedding a dynamic script language to the application can be a requirement, even for a Crystal application (eg: you need to allow hot reload of some business logic while keeping the low level compiled service running, or being able to gradually iterate on some moving parts running interpreted code until it’s stable enough to be merged in the compiled main engine, or being able to separate concerns in an application between two teams, one being low level the other high level, could be useful when you will develop your first game in Crystal once the windows support is added :).

If Crystal provides some facilities to seamlessly interact with interpreted Crystal code running in the same process (or another thread) and the other way around: getting the interpreted code being able to use objects/functions exposed by the main compiled app (handling boxing/unboxing, sharing memory for data structures ??) then this would be a great asset for Crystal. Is this something that will be possible with the upcoming Crystal interpreter ?

Also I’m wondering if the webassembly target for Crystal compilation is something that is considered for the future ? Porting the Crystal interpreter to webassembly could be an alternative but it would require the Crystal bytecode to be standardized, and will retain the slowness of still being interpreted vs llvm->wasm.

PS: thanks to all speakers at the Crystal conf, all topics were very interesting.

3 Likes

Hey! Glad you enjoyed the talks at Crystal Conf.

To address the WASM question, https://github.com/crystal-lang/crystal/pull/10870 This PR has been started. So there’s definitely movement on that front.

I’m not sure about the whole embedding the interpreter. My guess would be that if you required the right files in your app, you could probably make something work in terms of building a game with a front-end scripting.

I was concerned by the development iterations slowed down by compile times and also by the lack of convenient debugging facilities

Just on this note, in my case coming from the same dynamic world, I’ll say that I don’t miss those tools nearly as much as I thought I would. I found that I was using those sort of REPL and Pry like debugging to handle weird type cases where I wasn’t sure what type I was dealing with. These became a crutch. Once I started working with Lucky and Crystal, I had a better sense of relief that the object I was passing around did what I wanted, and I never needed to question it. Now, granted, there are times I do wish I had those sorts of tools, it’s just a LOT less than I originally thought.

I’d like to invite you to try giving it a shot instead of waiting. You might find that it’s an easier transition than you anticipate :smiley:

2 Likes

Hey amigrave!

Right now it’s about 3 to 10 times, or more, slower than Ruby. It’s pretty slow to be honest.

Right now I’m not coding the interpreter in that way. The only use cases I have in mind are:

  • being able to put debugger and inspect the program at that point (with next, step, etc.) working
  • as a repl, though right it doesn’t work quite well (for example if you run it and do require "http/client" it fails with an error)

So mostly just the first point for now.

The main issue with embedding the interpreter is that the type IDs used for types won’t match those in the compiled code, so you can’t really send objects from one place to another. This is the same issue as trying to use Crystal as an embedded C library: it just doesn’t work.

In summary, the interpreter won’t support the use cases you want, and I won’t be working on that because I’m doing it in my free time and those things don’t interest me, but of course anyone is free to try to make these things work. I’m only mentioning “me” because I’m the only one working on the interpreter right now, so chances of someone else doing this are very low.

Cheers!

1 Like

Yes, full interoperation between compiled and interpreted code might be hard and I’m not sure if it would really be that useful. But it’s not impossible and maybe we’ll see that some day.

However, I think it should be fairly easy to expose something like a C API to interact between the interpreter runtime and an external context (e.g. a compiled Crystal runtime).

1 Like

@jwoertink , @asterite , @straight-shoota Thank you all for your reply !

I’ll check if ducktape.cr fits my use cases.