Hi all,
I just came here to give some props to the Crystal project and everyone involved in pushing this forward, and also to share my and my team’s experience with porting a Ruby web app to Crystal over the last two years. Maybe this helps someone else who is on the fence of doing something similar, making a decision.
TLDR; We are using Crystal in production for a largeish enterprise project, and are very excited of how it improved the project. Of course there are some bumpers and downsides, but we are optimistic, that these will improve in the future.
Alright, so some background: we are a small team, developing and SaaSing a B2B web application for a rather large customer for more than 10 years now. Initially, the app was developed in Ruby, using the Merb Framework. If you have not heard of it, you are not alone. We were some early adopters of this MVC framework, which eventually got merged into Rails, leading to Rails 3. This left us with an unsupported framework, stuck with an unsupported Ruby version.
We were at the point, where an upgrade was inevitable. With increased complexity of the application, we were also hitting some performance bottlenecks then. This was when we decided that instead of a painful Rails upgrade, we could get a step further and move on to something more robust. Because of the similarity to Ruby and its promise about speed and performance, Crystal looked like a suitable solution.
After some proofs of concept, and having nothing learned from our Merb adventure, we decided to jump to the next esoteric technology, and started migrating Ruby code piece by piece to Crystal.
The Migration Strategy
We wanted to avoid developing a port in the basement for two years, and then blow everything up with a big bang release. Instead, we kind of applied a strangler pattern – subsequently replacing Ruby with Crystal code, running both technologies in parallel, until at some point, the Crystal code exceeds the Ruby code.
We did so by changing the architecture from a classic MVC pattern with server-side rendering to client-side rendered JavaScript components, talking to a (Crystal) backend API. The replaced Ruby views are merely empty scaffolds, serving only as hulls for the JavaScript/Crystal functionality. This allows us to run the legacy code and the new app in parallel.
This is pretty cool in multiple ways. There’s a comparably low risk when porting single modules, with the ability to roll back in case of problems. Also, you immediatly feel the performance difference when navigating from a Crystal backed part of the application to a legacy Ruby part.
Porting Ruby Code
Of course one of the main reasons why Crystal was considered in the first place is its similarity to Ruby and the promise of easy code portability. Indeed, once you know the usual suspects (includes?
, nil checks or adding not_nil!
, empty []
and {}
initialization, etc.) it is fairly straight forward to copy at least shorter methods just as they are.
Other things took some more effort, mostly related to Ruby meta programming. The application needs to be pretty configurable, and therefore is composed of lots of plugins, middlewares, hooks and whatnots, to customize single instances. In the Ruby part this happens a lot via reflection, based on strings from config files. At first this seemed as a show stopper for us and Crystal. But after some consideration we realized that, since we fully know the domain anyway, there is no need for this absolute flexibility in our case.
For example, this Ruby Code
plugin = Plugins.const_get(some_plugin_name).new
would get translated to something like this
def getPlugin(plugin_name : String)
case plugin_name
when "Plugin1"
return Plugins::Plugin1
when "Plugin2"
return Plugins::Plugin2
else
raise "Plugin #{plugin_name} not found"
end
end
plugin = getPlugin(some_plugin_name).new
If anything, this approach is way safer than the Ruby variant, as the value from the dynamic plugin name is implicitly whitelisted/sanitized, which probably did not happen everywhere in the Ruby parts before.
Jennifer as ORM
Following our habit of chosing the most niche technology, our choice regarding ORM was Jennifer. As we evaluated the other ORMs available at the time, each one had some downside which made it unsuitable for our situation.
While Jennifer looked pretty rough at first, it turned out to be the best match for us. Its query builder allows for nice API design, while letting you enough control and transparency of the generated SQL. I think using Jennifer and its proximity to raw SQL is responsible for a big part of the performance improvement we are experiencing.
However, we really had to learn our way with Jennifer. It has a lot of quirks you have to be aware of, and documentation is incomplete. There are lots of different (undocumented) ways to do the same thing, and we had to dive deep into the Jennifer source code regularly, to fully understand how to use it in certain situations. Errors caused by query DSL code result in You found a bug in the compiler
errors, instead of meaningful error messages.
Having that said, we wouldn’t have been able to come that far with the port, would it not be for this library, and it really became a very important part of the whole app.
Development Flow
One of the biggest changes in the overall way of working comes of course with the compile step. While I have a CS background, I haven’t compiled anything of considerable size during my time in professional web development in more than a decade. We are running and building our dev environments on VMs locally. Compilation of our monolithic app now takes at least three minutes, but this varies widely, and we are constantly scratching the limits of memory. Quickly punching in some debug output and hitting reload is not possible anymore, instead you really have to plan out things like that.
On the other hand, once you get the hang of it, you really appreciate the assurance you get from the type system and compilation, and returning to an untyped, interpreted language feels like walking a tightrope.
Current State and Overall Mood
The Crystal backend service now consists of around 26000 lines of code, and 3500 of whatever the compiler counts up to. We are now at a place where we can say that we have tackled and solved all the potential show stoppers we had on our radar, and the parts involving the most complexity have been ported to Crystal successfully. The improved performance is indeed remarkable, although I think that a good part of this also goes back to general refactorings we made during the process.
Compile times and memory consumption during compilation are the biggest concerns, and the most frustrating parts for now. In the worst case, we would have to split up the application service in the future, which makes for smaller chunks, but would increase complexity during deployment (microservices is nothing we are aiming for).
The reason why we can justify this whole move is, that the whole architecture change, untangling frontend and backend, was worth it by itself, regardless of whether we chose Crystal or a different technology for the backend. If for some reason we hit a dead end with Crystal at some point, we are at least not locked into an eco system, and porting back from Crystal to something else (Ruby? haha) would arguably be easier, since Crystal forced us to create cleaner and more robust code.
OK, this turned out a really long post, thanks for reading :) I just had the urge to share this kind of success story here, and again, show some appreciation for the whole Crystal project. I hope we can give back and support the project in the future, as we have seen the huge potential of it.