Incremental compilation exploration

Crystal Contrib
55

This topic got hot all of a sudden!

Thanks @asterite for providing your thoughts, and exploring so much of this space in the first place. It’s really educational, and your example helped solidify the challenges moving forward.

My own experience reflects @MistressRemilia’s - I come from an enterprise Java background where even simple Java projects take a noticeable amount of time to build (unless using an IDE, which is a requirement for Java development at this point, but doesn’t solve CI/CD now taking forever), and the build times of crystal are mere annoyances in comparison. My largest hobby crystal project clocks in at ~7300 lines of code, and takes ~55 seconds to compile from a cold start, usually ~30 seconds after that. I also don’t use an LSP at all (Sublime has pretty good prediction with regular regular expressions and I haven’t felt the need for one yet).

Also like @MistressRemilia, I run into an inflection point with new projects, where I’ll start with adding no typing information to anything, eventually get to a point of “Well dang, what was the type of this argument again? Sheep” (with potentially stronger language), and then react by adding types to everything.

I could see a future of Crystal development such that:

  1. New and small projects continue to build and develop as they are today
  2. This inflection point is encountered by a project when it gets $LARGE (for arbitrary values of $LARGE)
  3. Project adds typing information to everything, able to take advantage of incremental compilation, development continues.

The two crystal changes that I could see helping this (both suggestions were provided above, and I think both would be needed) are:

  1. A new crystal tool that adds missing typing information to methods after type inference is complete (would operate similarly to the existing format crystal tool in my mind) to quickly add typing everywhere it’s needed.
  2. A new compiler flag that would fail fast whenever it encountered a new signature that was missing typing information (rather than having multiple “tiers” of compilation speed, I’d prefer just to have a one-and-done approach).

I think with these, crystal would be a viable approach for larger projects going forward. It wouldn’t help with the LSP problem for smaller projects as much. I wonder if there’s a different approach that could be used for that?

8 Likes