Formatting Spec Test Results

JamesGood626 · December 8, 2021, 11:50pm

So I’m looking at this currently, and found myself wanting a pretty printed output of the data structure. I’m coming from Elixir, so I think about the way that the data structure is usually output in a way that’s readable without having to copy the “got” result and manually indent everything myself.

I tried looking at the spec documentation to see if there’s some configuration I can set to make this pretty printed format possible, and I haven’t been able to ascertain whether it is possible.
https://crystal-lang.org/api/0.35.1/Spec.html#add_formatter(formatter)-class-method

I also did a quick forum search before posting this, and I’m still at a loss.

If there’s something very obvious that I’m missing, I’d greatly appreciate it being pointed out.

Expected: “BOOM”
got: {{“#sapphire.off” => State::ExecutableNormal(@id=State::AbsoluteStateID(@id=“#sapphire.off”), @initial_state=nil, @on_entry=nil, @on_exit=nil, @transitions=[ExecutableTransition(@event=EventType(@type=“OFF_TO_ON”), @from=State::AbsoluteStateID(@id=“#sapphire.off”), @to=State::AbsoluteStateID(@id=“#sapphire.on”), @actions=nil, @guards=nil)], @child_states=nil, @invoke=nil), “#sapphire.State::RelativeStateID(@id="on")” => State::ExecutableParallel(@id=State::AbsoluteStateID(@id=“#sapphire.State::RelativeStateID(@id="on")”), @on_entry=nil, @on_exit=nil, @transitions=[ExecutableTransition(@event=EventType(@type=“ON_TO_OFF”), @from=State::AbsoluteStateID(@id=“#sapphire.on”), @to=State::AbsoluteStateID(@id=“#sapphire.on”), @actions=nil, @guards=nil)], @parallel_states=[State::AbsoluteStateID(@id=“#sapphire.charging_status”), State::AbsoluteStateID(@id=“#sapphire.playing_status”)], @invoke=nil)}, [State::AbsoluteStateID(@id=“#sapphire.on”), State::AbsoluteStateID(@id=“#sapphire.on”)]}

Coming back to this a few hours later, as I haven’t received a response as fast as I did the first time I posted something on this forum.

If I can create a parser which could pretty print format this, and it be considered an open source contribution to Crystal, please let me know.

I have a book I can reference, never written one before, but if I can contribute to the language in this way then that would be cool.

jgaskins · December 10, 2021, 2:13am

I agree that the output for nontrivial objects is not great. I’m not sure what would be ideal, but your idea of pretty-printing it could at the very least be an improvement.

You can pretty-print any Crystal object to stdout by calling pp object or you can get the pretty-printed output by calling object.pretty_inspect. I solved solved a similar problem in the Crystal Playground app (includes screenshots to show the impact):

github.com/crystal-lang/crystal

Pretty-print objects in playground inspector

crystal-lang:master ← jgaskins:playground-value-formatting

opened 03:55AM - 21 Jun 17 UTC

jgaskins

+12 -3

I'd noticed that complex objects in the playground inspector weren't very easy t…o read: ![Screenshot before this PR](https://user-images.githubusercontent.com/108205/27366041-a5036a42-5611-11e7-9e02-5679bb6c4821.png) So I hacked out something quickly to make them a bit easier on the eyes: ![Screenshot with this PR](https://user-images.githubusercontent.com/108205/27366045-a844ede8-5611-11e7-9b83-d9d3c3423fbc.png) Known issues: - This introduces a horizontal scroll to that table for very complex objects like the one I posted - Possible solutions: - Reduce the pretty-print width - Make only that column of the table scroll horizontally (I think this is possible, anyway) - Syntax highlighting (even just for class names, ivars, numbers, and strings, similar to Pry) would improve readability even further I noticed that the contribution guide says I should open an issue before a PR, but I'd already written the code by then. :-) I did want to get feedback on this before I go much further with it, though.

JamesGood626 · December 17, 2021, 3:21am

I checked it out, and it seems that what I’m looking for (more readable output of structs upon the printing of a failed spec test result), can be achieved by using that object.pretty_inspect as opposed to just object.inspect as is being used at this location in Crystal’s spec code:

github.com

crystal-lang/crystal/blob/6529d725a61ed0ff83f9d9efdd18bb7229c9cebc/src/spec/expectations.cr#L43

    
      
                  Expected bytesize: #{expected_value.bytesize}
                       got bytesize: #{actual_value.bytesize}
                  MSG
              end
          
          
    return <<-MSG
                Expected size: #{expected_value.size}
                     got size: #{actual_value.size}
                MSG
            else
              expected = expected_value.inspect
              got = actual_value.inspect
              if expected == got
                expected += " : #{@expected_value.class}"
                got += " : #{actual_value.class}"
              end
              "Expected: #{expected}\n     got: #{got}"
            end
          end
          
          
def negative_failure_message(actual_value)

I’ve never contributed to open source before, but being able to provide a function or something for how the expected and got results should be outputted could be one option.

straight-shoota · December 17, 2021, 8:21am

Maybe the spec printer should actually use pretty_print here. It’s defined on every type and falls back to #inspect. But types that implement it, benefit from the improved format.

JamesGood626 · December 20, 2021, 4:19am

I can follow the steps listed on the github page for crystal lang and make the change; however, I’ve only managed to get structs/classes to be printed in a formatted fashion using the pretty_inspect function.

I tried pretty_print after reading your message, but I receive this error:

error in line 71
Error: wrong number of arguments for ‘This#pretty_print’ (given 0, expected 1)

Overloads are:

Struct#pretty_print(pp)
Object#pretty_print(pp : PrettyPrint)

I looked into the docs and source code here:
https://crystal-lang.org/api/1.2.1/PrettyPrint.html

github.com

crystal-lang/crystal/blob/4e6c0f26e/src/pretty_print.cr#L9

    
      
          # This class implements a pretty printing algorithm.
          # It finds line breaks and nice indentations for grouped structure.
          #
          # ### References
          #
          # * [Ruby's prettyprint.rb](https://github.com/ruby/ruby/blob/master/lib/prettyprint.rb)
          # * [Christian Lindig, Strictly Pretty, March 2000](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.2200)
          # * [Philip Wadler, A prettier printer, March 1998](http://homepages.inf.ed.ac.uk/wadler/topics/language-design.html#prettier)
          class PrettyPrint
            protected getter group_queue
            protected getter newline
            protected getter indent
          
          
  # Creates a new pretty printer that will write to the given *output*
            # and be capped at *maxwidth*.
            def initialize(@output : IO, @maxwidth = 79, @newline = "\n", @indent = 0)
              @output_width = @indent
              @buffer_width = 0

But it appears that there’s no default printed provided, and that I’d need to construct one myself, as to what the PrettyPrint class is, but was unable to ascertain …

Making an attempt:
Perhaps the IO here can be passed to the PrettyPrinter class:

github.com

crystal-lang/crystal/blob/4e6c0f26e/src/io.cr#L207

    
      
          # io.print 1, '-', "Crystal"
          # io.to_s # => "1-Crystal"
          # ```
          def print(*objects : _) : Nil
            objects.each do |obj|
              print obj
            end
            nil
          end
          
          
# Writes *string* to this `IO`, followed by a newline character
          # unless the string already ends with one.
          #
          # ```
          # io = IO::Memory.new
          # io.puts "hello\n"
          # io.puts "world"
          # io.to_s # => "hello\nworld\n"
          # ```
          def puts(string : String) : Nil
            self << string

Didn’t work:
https://play.crystal-lang.org/#/r/cgte/edit

The example in the playground doesn’t change anything after making that edit.

I gave it a shot, and the … sentence is where I began the effort, figured I’d leave it as it at least shows that I made an attempt in the midst of making this forum post.

I did a ctrl + f for instances of ‘IO::Memory’ and ‘Memory’, both yielded the same amount of search results, so there’s no class definition for memory in that file.

After checking the top of the same file, I saw this:
require “c/stdio”

I know nothing about C bindings, but attempted to look for a c directory, closest thing I found was lib_c, and it has nothing related to IO.

So if there’s some alternative IO class instance which is supposed to be provided to the pretty_print method, then I’m unable to determine what exactly that may be at this point in time.

asterite · December 20, 2021, 9:02am

I think it’s called pretty_inspect. Can you try with that one?

straight-shoota · December 20, 2021, 9:52am

Yes, the public facing method is Object#pretty_inspect. Implementations are #pretty_print. The API is not very nice. I assume it’s probably caused by the Ruby heritage - Ruby doesn’t have method overloading, so you need different names. In Crystal, both methods could have the same name and receive different arguments. Documentation needs improvement as well.

I’m sorry that my comment seemd to have lead you deep down in a wrong direction.

JamesGood626 · December 21, 2021, 7:29pm

Submitted a PR: Format spec results with pretty inspect by JamesGood626 · Pull Request #11635 · crystal-lang/crystal · GitHub

Hmm… one check for the PR is failing:
Linux CI / check_format (pull_request)

I ran this command before making the commit:
ln -s scripts/git/pre-commit .git/hooks

JamesGood626 · January 11, 2022, 9:52pm

module DetectDiscrepancies
  extend self

  struct Tracker
    @good_expected_matched : String
    @good_got_matched : String
    @remaining_expected : String
    @remaining_got : String

    getter good_expected_matched
    getter good_got_matched
    getter remaining_expected
    getter remaining_got

    def initialize(
        @remaining_expected : String,
        @remaining_got : String,
        @good_expected_matched = "",
        @good_got_matched = ""
    )
    end
  end

  def match(t : Tracker) : Tracker | Tuple(String, String)
      re = t.remaining_expected
      rg = t.remaining_got
      gem = t.good_expected_matched
      ggm = t.good_got_matched

      non_empty = re.size > 0 && rg.size > 0
      if non_empty
          matches = re[0] == rg[0]
          if matches
              Tracker.new(re[1..-1], rg[1..-1], "#{gem}#{re[0]}", "#{ggm}#{rg[0]}")
          else
              expected_result = "#{gem.colorize(:green)}#{re.colorize(:red)}"
              got_result = "#{ggm.colorize(:green)}#{rg.colorize(:red)}"

              {expected_result, got_result}
          end
      else
          expected_result = "#{gem.colorize(:green)}#{re.colorize(:red)}"
          got_result = "#{ggm.colorize(:green)}#{rg.colorize(:red)}"

          {expected_result, got_result}
      end
  end

  # Only exposed API Function
  def detect_discrepancies(expected : String, got : String) : Tuple(String, String)
      detect_discrepancies(Tracker.new(expected, got))
  end

  def detect_discrepancies(t : Tracker) : Tuple(String, String)
      detect_discrepancies(match(t))
  end

  def detect_discrepancies(result : Tuple(String, String)) : Tuple(String, String)
      result
  end
end

Only using this for the eq spec expectation function at the moment, but really this would help quite a bit more than the output being all white, helps more than the pretty_inspect in my opinion.

Just posting this here because the PR I’ve posted hasn’t been commented on at all.

Topic		Replies	Views
What does the format command change? Help & Support	5	923	June 5, 2019
How output the spec use the documentation format? as rspec? Help & Support	2	160	May 11, 2023
Would it be possible to add an option for the indent size for the format tool? Help & Support	4	283	April 5, 2022
Spec example description Help & Support	7	539	April 5, 2020
[Discuss] Formatter always format one-line method to multi-line, is it necessary? Help & Support	6	405	February 21, 2023

Formatting Spec Test Results

Related Topics