What is the difference between the spec module and exceptions?

https://crystal-lang.org/reference/guides/testing.html

require "spec"

describe Array do
  describe "#size" do
    it "correctly reports the number of elements in the Array" do
      [1, 2, 3].size.should eq 3
    end
  end

  describe "#empty?" do
    it "is true when no elements are in the array" do
      ([] of Int32).empty?.should be_true
    end

    it "is false if there are elements in the array" do
      [1].empty?.should be_false
    end
  end
end

Compared to:

https://play.crystal-lang.org/#/r/6knb

begin
	start = Time.now
    puts [1, 2, 3].size == 3
	puts  ([] of Int32).empty? 
	puts ![1].empty? 
    puts "Test Time: (#{Time.now   - start}ms)"
rescue e
	puts e
end

The spec module is literally built to do testing, whereas the second example there, is a hacky way that kinda gives the same functionality. Exceptions are for catching runtime errors, not testing.

Benefits of spec

  • Easier to run - crystal spec
  • Easier to read - [1].empty?.should be_false
  • Gives nicer output of how many specs ran, what failed, etc https://play.crystal-lang.org/#/r/6knr
  • Literally designed to do this

EDIT: See https://crystal-lang.org/reference/syntax_and_semantics/exception_handling.html for more info on exception handling.

It sounds like rescuing is still sufficient. If an error happens, it’s caught (and the great thing about Crystal is the individual fibers have their own exceptions isolated). For example, if an exception occurred in a client’s handle_connection fiber, their fiber is rescued, the error is printed, and most importantly, the app stays alive.

I feel like spec is more for core developers who are creating PRs to test code that will be added to the language?

Well yea, exceptions/rescuing is for the actual runtime of the code to handle errors, while spec is for writing tests about the application. They aren’t really related.

@girng Using spec, if you have a.should eq(b) and that’s false, an exception is raised. In your example if you have puts a == b and that’s false, that doesn’t throw an exception.

Plus you don’t know what failed (you have to count the number of true and false printed).

But search “why is testing useful” in Google will probably give a more complete answer of all the benefits of testing (like that).

Yeah, I guess that makes sense for specific cases. But honestly, I’m still not convinced why I should use it in my code and create hundreds of “describe” and “it” blocks all over, when all my methods are rescued already.

If code is already rescued, I don’t understand why it needs to be re-tested.
Isn’t that what exceptions are for. If there is an exception, your code is wrong, lol.

A rescue is there to rescue exceptions that would happen during the execution of that method, not necessarily assert that the output of that method is correct.

For example
https://play.crystal-lang.org/#/r/6kpn

This way, if you have full test coverage on your code, if a change to method A unintentionally affects the output of method B, then your tests could catch it, and you could then fix it. It allows you to have confidence that your application is working correctly without manually testing everything after every change. Which is a powerful feeling, and makes updating/changing code a lot less stressful.

There are of course other types of tests that can be used. Best google the difference between unit, integration, and end to end tests for more information.

@Blacksmoke16

That exception does get raised though:

https://play.crystal-lang.org/#/r/6kps

You changed the 2nd param. In that case it raises which i covered on line 9 of the playground link.

But what do you mean by “but can still be tested to assert proper error is raised”?

If the exception is caught, isn’t that enough to “assert a proper error”?

Asserting that a correct exception class and message are raised. I can’t really think of a good example, but it would be most useful when using custom exceptions. In that say other code is dependent on the class and message of the raised exception. If either of those were to change, that logic would break and you would have a bug.

If you test the raised exception is of the correct type and has the correct message, its one less area that bugs can be introduced.

EDIT: https://play.crystal-lang.org/#/r/6kq9

In this example, I’m asserting that test method raises an exception of type MyException with message of Something bad happened. I’m also asserting that the priority, which is a custom ivar on the exception, is correct.

Of course this is a sample example, but you could imagine that a custom exception could get thrown within a method, and this would allow you to test that it gets raised properly, with the expected message and any custom properties set.

@gring in your very first example, if you have a puts output false and you have thousands of those checks: how do you notice one if false, and how do you know which one failed? Specs organized this. Please read somewhere else about unit testing, for example in Ruby. I’m not sure there’s much point repeating all that info here.

@girng It seems to me that we are answering the wrong question. Tests (specs) are not about verifying that your code is crashing or not. They are about to tell you that some of your future changes don’t break the expectations of your application interface and logic. Testing your application is a completely different discipline that requires some time to learn and master. It is about describing your API and verifying that for certain input you get the correct output and gives you and your team confidence about future changes.

I recommend you to go thru some crystal (or other languages) libraries and see how the specs are used here. For example my lib: https://github.com/schovi/baked_file_system/blob/master/spec/baked_file_system_spec.cr :)

And of course, read some generic know how about writing tests for your app https://www.google.com/search?ei=BsKhXOCsIK2FjLsPqpGp2AQ&q=test+driven+development&oq=test+driven+develop&gs_l=psy-ab.1.0.0i67l2j0i20i263j0j0i67j0l2j0i67l2j0.2045.4895..5795...1.0..0.68.1134.20…0…1…gws-wiz…0i71j35i39.o38J8MSa1NY

2 Likes

Understood. Thank you for the explanation, @schovi.