macOS BigSur: Crystal 1.0.0 is much slower than Ruby 3.0.0

Community
1

Yesterday I just downloaded crystal and ruby and compared the performance between them with the simple hello program. I am expecting crystal is much faster than Ruby, however it shows crystal is much slower than Ruby 3.0.

macOS Big Sur 11.2.3 @ iMac
Processor: 3.4 GHz Quad-Core Intel Core i5
Memory: 32 GB 2400 MHz DDR4

zhuangyan@iMac-2 ~/coding/crystal $ cat hello.cr
puts "Hello World!"
zhuangyan@iMac-2 ~/coding/crystal $ time crystal hello.cr
Hello World!

real	0m2.906s
user	0m0.905s
sys	0m0.575s
zhuangyan@iMac-2 ~/coding/crystal $ crystal --version
Crystal 1.0.0 (2021-03-22)

LLVM: 9.0.1
Default target: x86_64-apple-macosx

zhuangyan@iMac-2 ~/coding/ruby $ cat hello.rb
puts 'Hello'
zhuangyan@iMac-2 ~/coding/ruby $ time ruby hello.rb
Hello

real	0m0.143s
user	0m0.066s
sys	0m0.027s
zhuangyan@iMac-2 ~/coding/ruby $ ruby --version
ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-darwin20]
2

You’re including the compilation time of the crystal program in your comparison. A more fair comparison would be like: crystal build --release hello.cr && time ./hello.

1 Like
3

For some more boost, there is also the --no-debug compile option. I usually forget that myself, but saw it elsewhere recently.

4

Thanks a lot.

zhuangyan@iMac-2 ~/coding/crystal $ time crystal build --release hello.cr

real 0m0.936s
user 0m0.778s
sys 0m0.148s

zhuangyan@iMac-2 ~/coding/crystal $ time ./hello
Hello World!

real 0m0.904s
user 0m0.003s
sys 0m0.006s

5

np. That still seems high tho. This is what i got:

time crystal build hello.cr 

real    0m0.608s
user    0m0.645s
sys     0m0.140s

time ./hello
Hello World

real    0m0.009s
user    0m0.004s
sys     0m0.008s

As far as I know this would have a negligible effect on performance, but it would reduce the size of the resulting binary.

6

Ah, so --no-debug is more of a file-size optimization; makes sense. Thanks @Blacksmoke16 for the clarification. :slight_smile:

7

Run the hello binary a second time and post the time it takes. Mac OS does a binary check the first time it runs a binary, that’s why it’s slow.

1 Like
8

Use GitHub - sharkdp/hyperfine: A command-line benchmarking tool to benchmark:
hyperfine --warmup 5 'crystal run ...' 'ruby ...'

9

It’s also worth noting that simple hello world performance is far away from any real-world use cases and such a small program that any benchmark is easily derailed by any kind of external factors.
And I wouldn’t expect there to be much of a performance difference between Crystal and Ruby if you print hello world in a loop. The bottleneck is probably IO (i.e. how fast you can write to the console) for both.

2 Likes