I’ve been working on a project for a few weeks called Moku, a federated social network similar to Mastodon, written in Crystal. I have an example of it running at https://jgaskins.wtf/ — check out the public timeline (currently implemented as just showing every post in the instance’s public timeline all within a single page).
It’s still super early development (and the code is still a mess as a result), but it works pretty well. You can follow people from other instances, post, like, and boost. Replying isn’t built yet, unfortunately. Notifications could work if I just build the UI for it.
Mostly this was an experiment for 3 things:
- Build a social graph with an actual graph DB
- Mastodon uses Postgres, which is amazing and flexible and all that, but isn’t great with graph queries
- Most requests to Mastodon involve around 20 SQL queries, most of which could be coalesced into a single graph query
- Running the Neo4j Crystal driver against Neo4j’s new DBaaS (called Aura)
- Needs a little work because Aura uses clusters exclusively but the driver is designed around single connections, and your DB subdomain doesn’t guarantee that you connect to a writable node in the cluster
- Basically, it means I need to build up a unified interface for both clusters and connections the way the official drivers use
- Building a social app using pure server-rendered HTML rather than a SPA like Mastodon and Twitter use
- I don’t know if this is something that will stick around but I really like it
- Coming back to the app on a mobile browser will put you back where you were, not at the top of the feed
- Liking/boosting a post does take you to the top of the page but the timeline is paginated so it’s not too far from where you were
I compared its performance to Mastodon for giggles by repeatedly hammering the home timeline. Both apps running within the same Kubernetes cluster). The difference is hilarious.
Mastodon uses 61.7% CPU just for the Rails server process and has local maxima of around 80%. Moku reached a whopping 1.1% local maximum and averaged around 0.7% CPU for the same workload. I think a big part of Mastodon’s performance woes are in ActiveRecord. With the Crystal Neo4j driver, I don’t even use an ORM. I map nodes in the DB to objects in memory, certainly, but I use query objects to fetch data and make use of streaming results.
Memorywise, Mastodon’s web process alone took up 470MB, but it also uses a Node.js websocket server and a Sidekiq process, which bumps the Mastodon installation up to about 800MB of RAM consumed. Moku uses 25MB. Because it’s still early development, I can just use Crystal fibers as a stopgap for background jobs so I don’t need to mess with message queues yet, but I do expect that memory consumption to grow (omg, it could be FIFTY MB) once I build that out for durability’s sake.
It’s also not using a web framework. I’ve built a mixin that lets me handle routing similarly to Roda and sessions are stored in Redis. Rendering is done as partials, I’m not even using a CSS preprocessor (or even CSS files) yet. That’s all just hardcoded into the HTML currently. I’m enjoying the minimalist development approach here. Maybe as the project grows I’ll be less interested in that and want to move to more conventional practices, but I’m finding this works really well for now.
Anyway, I’d love to hear y’all’s thoughts on the project. This is my first time using Crystal to do anything HTML-based. I’ve mainly been using it for back-end services, passing around JSON and Protobufs over HTTP and RabbitMQ.