Crystal, Rust, .NET, Swift, and JS relative performance

For the past few days I’ve been seeing some benchmarks floating around Twitter and the fediverse.

There were some others, but these two were the ones I saw the most.

Saving you some clicks: other than JS, they’re all within a few percentage points of each other. In all cases, the benchmark was a web app that both receives and responds with equivalent JSON payloads. I suppose the idea was to benchmark something more substantial than a “hello world” benchmark. I still don’t think that’s enough work to benchmark, but we’ll get to that in a minute.

My understanding is that C# and .NET are used interchangeably. I’ve never worked in that ecosystem, though, so feel free to check me on that.

I ran some benchmarks of my own to see how well Crystal fares because I’m always interested in the relative performance. Now, since these two folks have effectively shown that Swift, .NET, and Rust all perform pretty closely on this particular benchmark, comparing to one of them also compares to the others.

On my machine, the Rust implementation (copy/pasted from the gist linked in the tweet) handled 97k RPS across 1M total requests:

  Total:	10.3051 secs
  Slowest:	0.1218 secs
  Fastest:	0.0000 secs
  Average:	0.0003 secs
  Requests/sec:	97038.9722

This is great. It’s incredibly fast. This gives a baseline for comparison to the Crystal implementation … which does 98k RPS for the same total number of requests (with -Dpreview_mt — important because the other implementations were also using multiple CPU cores):

  Total:	10.1531 secs
  Slowest:	0.2014 secs
  Fastest:	0.0000 secs
  Average:	0.0003 secs
  Requests/sec:	98492.1149

This means that Crystal is as fast as Rust, .NET, and Swift at parsing and emitting these particular JSON payloads. But this isn’t a realistic workload. Parsing and emitting JSON is part a realistic workload, but to dial up the realism a bit, you should at least talk to some kind of data store.

I augmented the Rust benchmark to make a single Redis query while handling the request. This can simulate persisting the data being sent from the client, sticking the data into a queue to be processed asynchronously, or really anything beyond simply transforming one JSON payload into another — there are more efficient ways to do that than sending an HTTP call.

The code is here.

With only that change between them, here are the results from the Rust implementation:

  Total:	24.4925 secs
  Slowest:	0.0704 secs
  Fastest:	0.0001 secs
  Average:	0.0006 secs
  Requests/sec:	40828.8615

And the Crystal implementation:

  Total:	16.8755 secs
  Slowest:	0.0659 secs
  Fastest:	0.0000 secs
  Average:	0.0004 secs
  Requests/sec:	59257.6096

Crystal was on par with Rust when only parsing/emitting HTTP and JSON, but as soon as I added a single data store query in there, Crystal pulled ahead by almost 50%. Even the latency distribution was similar.


Latency distribution:
  10% in 0.0004 secs
  25% in 0.0005 secs
  50% in 0.0006 secs
  75% in 0.0007 secs
  90% in 0.0008 secs
  95% in 0.0009 secs
  99% in 0.0013 secs


Latency distribution:
  10% in 0.0002 secs
  25% in 0.0002 secs
  50% in 0.0003 secs
  75% in 0.0005 secs
  90% in 0.0006 secs
  95% in 0.0007 secs
  99% in 0.0015 secs

Previous notable performance benchmarks I’ve done between Crystal and Rust: