Hello, I’m doing a small proxy, I don’t know if I risk overflowing RAM when downloading large files because the browser is slower to download the file via Internet than the server via internal network (because the bandwidth). Does IO.copy handle back pressure? I have the impression that yes becau…

Hi! I don’t know what “back pressure” means in this case, but here’s IO.copy source code: crystal-lang/crystal/blob/dd40a2442fa186add8a82b74edb14a90aa1dae05/src/io.cr#L1116-L1124 def self.copy(src, dst) : Int64 buffer = uninitialized UInt8[4096] …

Thanks. The source is res.body_io, if it is read (copied to response.output) too slowly, do you think res.body_io will grow (in RAM) or is it limited via a buffer too? [image] asterite: I don’t know what “back pressure” means in this case [image] Backpressure explained — the flow o…

Yes, body_io is limited by a buffer. In general when using IO in Crystal there are no memory allocations involved.

This is great, thank you :+1: For info, I’ve done a lot of tests with 250MB files in parallel, it seems very stable, the memory grows and shrinks during uploads. It does the job. In case I’m not looking well or some other contraindication, I prefer to ask :slight_smile:

IO.copy(src, dst) handles the back pressure?

Help & Support

nico July 8, 2021, 6:14am 3

Thanks.
The source is res.body_io, if it is read (copied to response.output) too slowly, do you think res.body_io will grow (in RAM) or is it limited via a buffer too?

Topic		Replies	Views
High memory usage copying data from a `HTTP::FormData::Part` into a `IO::Memory` Help & Support	4	308	July 1, 2025
IO.copy buffer too small? Crystal Contrib	24	2735	September 20, 2022
Working with large bin files, proper way to do it? Help & Support	1	440	March 17, 2019
How to pipe context.response.body = HTTP::Client.get(url).body?	6	1211	June 12, 2020
Read ouput line by line from bash command(process bigfile) in Linux Help & Support	12	1719	September 9, 2019

IO.copy(src, dst) handles the back pressure?

Related topics