# The true cost of parsing Crystal source code

**URL:** <https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185>\
**Category:** Crystal Contrib\
**Created:** [June 26, 2025, 9:03pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185 "2025-06-26T21:03:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![HertzDevil](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hertzdevil/32/1023_2.png) [@HertzDevil](https://forum.crystal-lang.org/u/HertzDevil)\
**Post date:** [June 26, 2025, 9:03pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/1 "2025-06-26T21:03:43Z")

</div>

Passing `--stats` to Crystal when building or running any source code gives you a list of compilation stages and their times elapsed, the first stage being `Parse`. Most of the time, this stage takes less than 1 _millisecond_, even when rebuilding a large codebase like the compiler itself. This is naturally impossible, no matter how fast your computer runs.

The reason here is that [the scope of the stage is very small](https://github.com/crystal-lang/crystal/blob/34875ff3fce637b73712090eba4375bc96d80498/src/compiler/crystal/compiler.cr#L294-L311). It measures only the time taken to parse the entry points given on the command line, not the time taken for required files; they are parsed as [`Require` nodes are visited by the top-level visitor](https://github.com/crystal-lang/crystal/blob/34875ff3fce637b73712090eba4375bc96d80498/src/compiler/crystal/semantic/semantic_visitor.cr#L87-L104), meaning all that elapsed time counts towards the `Semantic (top level)` stage instead. Knowing that all the parsing goes through `Crystal::Parser#parse`, here is a rudimentary patch that prints the total time taken for all invocations of that method:

```crystal
module Crystal
  class Parser
    class_property elapsed = Time::Span.zero

    def parse
      t0 = Time.monotonic
      begin
        previous_def
      ensure
        t1 = Time.monotonic
        Parser.elapsed += t1 - t0
      end
    end
  end

  class Compiler
    private def print_codegen_stats(units)
      return unless @progress_tracker.stats?
      puts "Parse total: #{Parser.elapsed}"
      previous_def
    end
  end
end

```

We then build an empty file with `--stats`, meaning the prelude is solely responsible for all the parsing. On Windows, a typical run looks like this:

```auto
Parse: 00:00:00.000625400 ( 1.16MB)
Semantic (top level): 00:00:00.371690400 ( 129.64MB)
Semantic (new): 00:00:00.001411500 ( 129.64MB)
Semantic (type declarations): 00:00:00.022352500 ( 129.64MB)
Semantic (abstract def check): 00:00:00.008010900 ( 145.64MB)
Semantic (restrictions augmenter): 00:00:00.006648800 ( 145.64MB)
Semantic (ivars initializers): 00:00:00.095378200 ( 145.70MB)
Semantic (cvars initializers): 00:00:00.003065200 ( 145.70MB)
Semantic (main): 00:00:00.212487200 ( 193.82MB)
Semantic (cleanup): 00:00:00.000268600 ( 193.82MB)
Semantic (recursive struct check): 00:00:00.000578700 ( 193.82MB)
Codegen (crystal): 00:00:00.199303900 ( 193.82MB)
Codegen (bc+obj): 00:00:00.182948200 ( 209.82MB)
Codegen (linking): 00:00:00.714077300 ( 209.82MB)
Parse total: 00:00:00.268039300

```

0.27 second, or over 70% of the top-level phase, is spent on parsing source code. It may seem little compared to the main and codegen phases, but definitely contributes to the slowless perceived by some newcomers running their first Hello World program and other short scripts. It also dominates compiler components that only require the top-level semantic pass, like the docs generator and the hierarchy tool. Below is a flame graph showing the same symptom: (this is done on Debian WSL)

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/crystal_lang/original/2X/7/7d493c83264828b013cf947dfdc4353596367278.jpeg)

The graph is focused on the outermost stack frame involving the top-level visitor, and all the highlighted frames are instance methods of `Crystal::Parser`. They indeed occupy the majority of the top-level phase.

Traditionally, much of the work on improving compilation times has been directed at the codegen phases, and there was little attention to the parser. There are probably many missed optimization opportunities here, and we may want better reporting facilities for `--stats` too, not least because the stages of interest are not perfectly sequential, i.e. parsing could happen in the middle of the main phase too for macro methods referring to the `@type` macro variable.

---

<div class="post-metadata">

**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\
**Post date:** [June 27, 2025, 7:30pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/2 "2025-06-27T19:30:54Z")

</div>

Yeah, I guess it’s time to end the tale of the compiler’s incredible parse efficiency 😆  
Thanks for digging into this.

Still, parsing consumes only a small portion of the overall compile time (I guess ~10-15%).

Another consideration is growth behaviour: Parsing scales strictly proportional to the size of the codebase. But the complexity of semantics (and, to a lesser extent, codegen) grows exponentially.  
As a result, improvements in parsing have less effect on the overall compilation time for large codebases than smaller ones.

This might put a bit of a perspective. But doesn’t mean we should not try to optimize the parser. Any improvements are helpful. Especially if there are any low-hanging fruit.

---

<div class="post-metadata">

**Author:** ![Xen](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/xen/32/2288_2.png) [@Xen](https://forum.crystal-lang.org/u/Xen)\
**Post date:** [July 6, 2025, 12:03pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/3 "2025-07-06T12:03:16Z")

</div>

> [@straight-shoota](#):
>
> As a result, improvements in parsing have less effect on the overall compilation time for large codebases than smaller ones.

Ah, but each `crystal spec` run is a compilation, and if you structure your specs in proper bite size, the cumulative effect of better small code base compile times will have a significant impact on DX experience if you run just the spec(s) for the part you’re working on rather than the whole suite.

So might be worth expending some effort?

---

<div class="post-metadata">

**Author:** ![Barney](https://avatars.discourse-cdn.com/v4/letter/b/48db29/32.png) [@Barney](https://forum.crystal-lang.org/u/Barney)\
**Post date:** [July 6, 2025, 7:57pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/4 "2025-07-06T19:57:27Z")

</div>

In the compile example given by Hertzdevil, parsing (0.27s) took 9.5% of the compilation time. (2.81s total).

In any real application, no matter how small your specs are, total time is going to be longer than this, but the parsing won’t. Meaning it’s always going to take \<9.5% of the total build time.  
Assuming that the time it takes to parse the source could be reduced to 0 seconds, you’d be saving 0.27 seconds, which is hardly something you’d notice in anything other than a simple `hello world` or scriptlet 😕

---

<div class="post-metadata">

**Author:** ![ysbaddaden](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/ysbaddaden/32/2252_2.png) [@ysbaddaden](https://forum.crystal-lang.org/u/ysbaddaden)\
**Post date:** [July 7, 2025, 8:42am UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/5 "2025-07-07T08:42:41Z")

</div>

@Barney The total build time is 1.8s, then half of it is the LLVM codegen + link time (0.9s) where we can’t do much, so 0.27s is in fact 29% of the total time in crystal.

---

<div class="post-metadata">

**Author:** ![Barney](https://avatars.discourse-cdn.com/v4/letter/b/48db29/32.png) [@Barney](https://forum.crystal-lang.org/u/Barney)\
**Post date:** [July 7, 2025, 6:48pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/6 "2025-07-07T18:48:39Z")

</div>

2 maths degrees and I still can’t use a calculator 🙈 Thank you for correcting me, that sounds like a more substantial difference.

---

<div class="post-metadata">

**Author:** ![mjblack](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mjblack/32/1878_2.png) [@mjblack](https://forum.crystal-lang.org/u/mjblack)\
**Post date:** [July 13, 2025, 4:14pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/7 "2025-07-13T16:14:06Z")

</div>

what was used to generate that graph?

---

<div class="post-metadata">

**Author:** ![HertzDevil](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/hertzdevil/32/1023_2.png) [@HertzDevil](https://forum.crystal-lang.org/u/HertzDevil)\
**Post date:** [July 13, 2025, 4:17pm UTC](https://forum.crystal-lang.org/t/the-true-cost-of-parsing-crystal-source-code/8185/8 "2025-07-13T16:17:33Z")

</div>

Debian `linux-perf` and:

```sh
$ make clean crystal release=1 FLAGS=--frame-pointers=always
$ perf record -F 1000 -g -- bin/crystal build --no-codegen src/compiler/crystal.cr
$ perf script report flamegraph -o flamegraph.html --allow-download

```