When you’re working on a specific piece of code and want to run tests for it (often repeatedly in an iterative process), the spec library has options to filter which tests you want to execute. For example with --location <file>:<line> you can specify to run only the example that contains the given location.
But this filtering happens at runtime. The compiler still builds the entire spec suite, and we’re just selecting to run only a few of the examples.
If we could however avoid building the unused examples in the first place, that would be less code to build, a faster response cycle.
One way to do that is to comment out all the specs you’re not interested in. But that’s tedious work. It’ll rarely pay off.
What if we could do this automatically though? The typical way to run specs is with crystal spec which first builds the spec binary and then executes it. So at build time we already know which locations to filter.
For the runtime filtering we’re utilizing the __FILE__, __LINE__ and __END_LINE__ magic variables which can provide information about a methods call site when assigned as default values of method parameters.
The same mechanism works with macros to! If we turn the spec methods into macros we can apply the filtering at compile time and avoid building unused examples.
A simple PoC which receive the line numbers from an environment variable:
FILTER_LINES = {{ env("SPEC_FILTER_LINES").split(",").reject(&.empty?).map(&.to_i) }}
{% verbatim do %}
macro it(description = "assert", file = __FILE__, line = __LINE__, end_line = __END_LINE__, focus = false, tags = nil, macro_line = __LINE__, macro_end_line = __END_LINE__, &block)
{% if FILTER_LINES.any? {|l| macro_line <= l <= macro_end_line} %}
::Spec.cli.root_context.it({{ description }}.to_s, {{ file }}, {{line}}, {{end_line}}, {{focus}}, {{tags}}) do
{{ yield }}
end
{% end %}
end
{% end %}
This works. It can be a stand in for the it spec method.
Of course this is quite limited, but it could already prove useful when running a single spec file. But we’d need a few more enhancements to make it really good. For example taking the file name and example groups (describe, context) into account.
Then we could probably expand the same principle to filtering by name and tags as well. All the information is available at compile time.
The additional macro evaluations obviously have some compiler cost as well. But they make that up by skipping unnecessary code entirely.
Still we wouldn’t want to pay that price when we really want to build the entire spec suite. But that’s no problem! If there are no filters, we can skip the entire filter logic. it can be the simple method it is right now.
I’m just dumping this idea hear, to see if it seems like something worth persuing.
The prototype looks promising, but I’m sure there’ll be challenges in many of the details. For example I’m not sure if the macro/method swap would work seamlessly in practice.
Another issue, as you might’ve noticed, I’m using extra parameters macro_line and macro_end_line because sometimes calls assign runtime values (most commonly forwarding the original location in a spec helper).
I don’t think the compile time filtering can work in these cases unless we make all the spec helpers macros as well. 