Accessing HTTP::Server::Response body

In some cases (mainly testing), you may want to access the bytes that were written to the HTTP::Server::Response, but as of right now, there’s no way (that I’ve found) to do that.

It looks like this topic was discussed on Gitter a while back. I recently just ran in to this when trying to write a spec in Lucky. It seems even Athena has added in a patch to do this.

The general idea here is that you’re testing that certain things are written to the response. So you build out your mock context, run some fancy methods which may call response.print(data) internally, then you’re testing that everything came out the way you expected.

In my case, we had a Lucky spec that was testing that nothing is written to the body when making a HEAD call… well… it turns out that the spec did

make_head_call
context.request.body.to_s.should eq ""

:grimacing:

Once I patched the response per @Blacksmoke16’s suggestion, the spec failed (as it should have).

I imagine there’s a good reason for not being able to read the response output, so I’d like to propose just adding some documentation to HTTP::Server::Response - Crystal 1.2.2 with an example on how to change the output to IO::Memory, or something along those lines. It could mention “Use this when testing the response body”.

2 Likes

In theory all you need to do is:

memory = IO::Memory.new
context.response.output = IO::MultiWriter.new(context.response.output, memory, sync_close: true)

However it seems that currently doesn’t work unless you explicitly do context.response.flush at the end. We should probably fix IO::MultiWriter to change that.

3 Likes

TIL there’s a thing called MultiWriter :joy:

2 Likes

I will say in regards to the Athena use case I’m not sure this would work. Athena uses this for its integration tests so you can do like: self.request("GET", "/some/endpoint/").body.should eq "some response". I.e. the user wouldn’t have access to the memory IO since the server object is created from scratch within the integration testing code.

Even if you did, you then run into the issue of that HTTP::Server::Output has the entire response written to it, not just the response body:

HTTP/1.1 200 OK
cache-control: no-cache, private
date: Sat, 13 Nov 2021 15:17:54 GMT
content-type: application/json
Content-Length: 15

"some response"

Which then you’d have to to string manipulation to get the response content and you lose the other information from the response, like the headers/status etc. So in the end i went with that monkey patch as it provided the best API.

One of the things I do to unit-test routes in my apps using Armature without the actual HTTP server machinery is to build an HTTP::Client::Response from that serialized response.

require "http"

response_body = <<-EOF
HTTP/1.1 200 OK
cache-control: no-cache, private
date: Sat, 13 Nov 2021 15:17:54 GMT
content-type: application/json
Content-Length: 15

"some response"
EOF

response = HTTP::Client::Response.from_io(IO::Memory.new(response_body))

response.body # => "\"some response\""

It’s pretty neat because it pretends to be an HTTP client talking to an HTTP server but there’s no actual HTTP server in between.

I really should publish it as a shard, now that I mention it. It doesn’t do anything specific to Armature and its only dependency is the standard library. You call methods like get(path, headers, body), just like with HTTP::Client, and you can build abstractions on top of it to handle serialization formats like JSON or HTML forms.

1 Like

Alrighty, I published it.

@jwoertink If you’re doing this in a Lucky app, I believe you can do something along the lines of this:

require "hot_topic"

client = HotTopic.new(Lucky::RouteHandler.new)
client.get("/foo")

And it will invoke your Lucky app similar to Lucky::BaseHTTPClient except it won’t go over a TCP socket.

4 Likes

In my opinion, such a tiny test helper should rather become a part of lucky than a standalone shard which essentially provides a single, relatively simple method.

I actually think that such a test helper should be in the standard library. Then anyone can use it, and it will keep working even if we change how http server is implemented internally. And we could use it to test our existing handlers.

I think the standard library should provide more testing libraries.

7 Likes

I was thinking it’d be good for the stdlib, too. I can put together a PR to discuss it more.

3 Likes

I didn’t look at the code and just assumed it’s something specific to lucky. But if it’s generic, I think it would be a great enhancement for stdlib and could even simplify some of our own handler specs.

I agree that we should have more spec helpers available for tests in user code.

2 Likes

That’s pretty awesome! Thanks for sharing

2 Likes

It’s just 33 lines of code, and that includes the version number. The impmemention is incredibly simple, which is a great thing :-)

6 Likes

this really neatly solves something i’ve wasted too much time thinking about.
can’t wait to utilise it!

thanks :star2: @jgaskins :star2:

1 Like

PR is up

3 Likes

nice! :rocket: