Has anyone started working on a Ruby => Crystal codemod / transpiler / converter?

I’ve read the Crystal for Rubyist wiki page, and this Reddit comment about some of the challenges.

But I can also think of a lot of examples that could be easily automated and save some time:

  • Rename all file extensions from .rb to .cr
  • Translate methods / operators with different names, e.g.:
    • include? -> includes?
    • key? -> has_key?
    • detect -> find
    • collect -> map
    • respond_to? -> responds_to?
    • length, count -> size
    • __dir__ -> __DIR__
    • and -> &&
    • or -> ||
    • require_relative "foo" -> require "./foo"
  • Replace single quoted strings with double quotes

Then run the Crystal type-checker + formatter + ameba on the result and start fixing the remaining issues.

I was surprised that I couldn’t find anything on Google or the Implementations/Compilers section in awesome-ruby.

I use a couple of very small Ruby gems that should be very easy to port to Crystal, such as this bad_words_detector. So I’m interested in working on a codemod/translator to help with some of these cases.

I think the easiest way might be to fork rufo and make it generate semi-valid Crystal.

2 Likes

I decided to fork rufo and created a ruby_crystal_codemod repo on GitHub.

Haha I just started working on a few of the easy changes, and then @asterite popped up in the git history!

2 Likes

:laughing:

I created rufo! It was previously under my github account but I eventually gave up: maintaining one big open source project is already a lot of work! :slight_smile:

I think translating Crystal syntax to Ruby is a good idea, I’m curious about the result. Doing it without a semantic pass might break in some cases. For example what if I do have a detect method on a custom class and you go ahead and rename it to find? Then it won’t work.

Also, the std is similar but not exactly the same so there might be some friction there too.

Another thing to consider is macros: in Crystal they work at the AST level and there’s no way to translate that to Ruby… unless you actually run the semantic phase to expand macros, but that also involves typing the entire program.

So one idea would be to type-check crystal, then instead of generating LLVM code you would generate Ruby code… but it gets tricky with things like inline assembly, or methods that should invoke Ruby methods instead of being translated to Ruby (for example String, Array or IO methods… or anything that uses pointers).

3 Likes

It’s a bit scary to see that already more than 2 years passed… what is going on with time??

6 Likes

Oh that’s awesome, I didn’t realize that!

I’m also interested in converting Crystal syntax to Ruby, and I posted about that here. My idea was just so that I could run some simple Crystal code using Ruby’s interpreter, so I would have access to a real REPL where I can modify variables and define new methods.

But in this case I’m trying to see if I can go the other direction and transpile some Ruby code into the Crystal syntax. I think this could make it easier to port some simple Ruby libraries.

A semantic pass would be really great, but I don’t know if that would be feasible in a Ruby codebase without any type annotations. But I could probably do a simple regex search to see if any of the source files contains def detect, and then show a big warning in the output.

I have a really simple proof-of-concept working now! https://github.com/DocSpring/ruby_crystal_codemod/tree/master/example_code

It just handles the following cases:

  • Rename all file extensions from .rb to .cr
  • Replace single quoted strings with double quotes (default in rufo)
  • require_relative "foo" -> require "./foo"
  • __dir__ -> __DIR__

And now Ruby and Crystal can both run the code and produce exactly the same output.

Haha but my implementation is extremely bad. I kept trying to find some way to reach into the next tokens and modify them, but I couldn’t get anything to work. So I just resorted to this crazy @wait_for_consume_token_count_then_set_relative_path counter to get something working.

Is there any easy way to update the path string in #visit_command and add ./?

When I give talks on Crystal this idea comes up a lot. It would be really nice to point people to something like this even if it only covers basic libraries.

1 Like

Wow I just realized that I could translate many Sorbet type annotations into Crystal type annotations! I’m definitely going to add that to the roadmap. I’d love to make it as easy as possible for some of the engineers at these companies to start looking into Crystal:

if anyone is interested the repo is here.

3 Likes

I’m not confident that this is a good idea, at least not until Crystal passes the 1.0 point. Like many people I started out thinking it would be great to convert programs between ruby and crystal. At first you pick a few small-ish programs and it seems like this would be easy and very useful.

But as I tried to convert larger and more complex ruby programs to crystal, I found that for some programs you’ll really want to rethink what your program is doing. Maybe you can get it running, but in some way which is much worse that what you’d get if you rewrote it by hand.

And the people who would be the most interested in this are people who know ruby well, but haven’t learned all of crystal. That means once they have some initial program running, they’ll be tempted use this sub-optimal crystal program as a template for new crystal programs that they write.

I am sure this kind of transpiler will always be very tempting, but based on my experience you’ll soon get annoyed at all the little details that you have to handle to make a correct and reliable transpiler, and you’ll switch your focus on other projects.

2 Likes

@drosehn You’re probably right, but it’s been a lot of fun to work on! I think I will probably keep working on as a hobby project / academic exercise to learn more about the Crystal language.

I added this special #~# BEGIN / END comment syntax (inspired by the Rufo test suite!), so it’s now possible to write Ruby and Crystal in the same file.

I got most of the easy transformations done, but I’m really excited about the idea of transforming Sorbet Type Annotations.

I’m also really interested in the idea of automatically instrumenting Ruby code to keep track of all the variable assignments, and building up a list of all the possible types that it sees at run-time. Then I could pass that to the transpiler so that it converts them into type annotations. (Maybe using Sorbet annotations as an intermediate step, so that people can also use this for Sorbet.)

Although I don’t expect to directly translate a full app it could lead to a translator assistant for snippets and scripts.

It can help for users migrating from Ruby to Crystal to learn the delta in a more interactive way rather than reading.

It might be more interesting if the criteria of the diffs are available at hand and not only the changes in the code, but it’s still valuable IMO.

1 Like