Very slow build speeds for hello world

Help & Support
41

I didn’t use the -s option much before, but when I tried it, I was impressed by the nice output.

However, the -s option doesn’t show that most of the time is spent on two specific LLVM functions. These functions are each called only once. Most of the Crystal compilation time is spent on LLVM, and Crystal itself does very little.

This means speeding up the Crystal compiler might not reduce build time much. To improve build time, we might need to make LLVM faster or change how Crystal outputs LLVM-IR. For example, we could split it into parts for parallel compilation. But I’ve heard that splitting Crystal might cause type inconsistencies because the type number for each class changes with each compilation.

If you have been using Crystal for a long time, you probably know this. But newcomers often wonder why builds are so slow. Understanding that most of the time is spent on LLVM, not Crystal, will help them understand and feel comfortable.

3 Likes
42

AMD Athlon™ X4 840 Quad Core Processor 3.10 GHz 8G windows 11

test!

Command Mean [s] Min [s] Max [s] Relative
crystal run ./hello_word.cr 3.647 ± 0.397 3.152 4.278 1.00
43

You might want force a full rebuild if you’re doing it multiple times. Otherwise it should be mostly incremental.

1 Like
44

I wrote a blog post about this benchmark on dev.to.

2 Likes
45

I learned that LLVM applies various passes to LLVM-IR to perform optimizations. It would be kind of interesting to visualize which passes are applied, in what order, where they are applied, and how long each one takes. That said, this is a bit beyond my capacity, since I’m just doing it as a hobby.

46

So, I gave it a quick try on Google Colab. It looks like ModuleInlinerWrapperPass and DevirtSCCRepeatedPass are the heavy ones. They disappear at -O0, but the pattern is similar at -O2 (it’s just a bit faster).

require "kemal"

get "/" do
  "Hello World!"
end

Kemal.run

crystal build --release --emit llvm-ir server.cr

opt-15 -time-passes -O3 server.ll -o /dev/null 2> server.opt.txt

Bar chart and treemap by Google Colab’s Gemini. I did absolutely nothing🙃

image
image1189×989 87 KB

newplot
newplot935×525 26.6 KB

CRYSTAL_CACHE_DIR=/content/cache1 crystal build -O3 --verbose server.cr

newplot(1)
newplot(1)1020×525 72.3 KB

Colab Notebook Share Link

2 Likes
47

Those results are based on LLVM 15. The optimization pipeline and results are different on my local LLVM 20. It seems Crystal avoids premature optimization and bets on LLVM’s progress instead. The right approach, I think.

48

I am pretty sure it was Stripe that created Sorbet. I think Shopify uses different tools for type checking.

49

Stripe did create it, but Shopify does use it extensively. I worked there until last year, though if you’d prefer not to trust strangers on the internet (a wise choice, tbh), you can also confirm by looking through the contributors from the last 6 months: of the top 10, 2 are from Stripe and 5 are from Shopify. The other 3 might also be from either of those companies, but it’s not readily apparent.

1 Like