I’m getting a
GC Warning: Repeated allocation of very large block (appr. size 536875008): May lead to memory leak and poor performance
error when reading in a large json file (190mb) from
ARGF, passing it to a
Process.run, and puts the output.
input = IO::Memory.new output = IO::Memory.new error = IO::Memory.new IO.copy(ARGF, input) input.rewind Process.run("jq", ["."] of String, input: input, output: output, error: error) puts output.to_s puts error.to_s
Steps to reproduce:
- Install jq
- Get the json file
- Build the code
crystal build --release test.cr
- Run it
./test citylots.json > test.json
I’m not sure if this is an issue with my code, Crystal, or that because I’m going through a Process. Is there anything I should do about it or is it just a harmless warning? Or is there a more efficient way to handle this?