Hi
I’m brand new to crystal, and opted to write a Kemal project over the weekend as a learning exercise. It’s a fairly basic app that reads a flurry of inbound JSON post messages (about 40/s) into a nested struct (with include JSON::Serializable at the tops of those, and using ‘property’) and then the struct is pushed into a Deque to a limit of 1000 items. There are other api methods to read from the Deque to process the data and return results. I’ve compiled it with just ‘crystal build’ and run the exe on MacOs. Generally it works well. It consumes around 300MB RAM and peaks at 56% CPU usage. I’ve noticed it only uses 1 thread.
However I noticed it randomly crashed out with a ‘Too many open files (File::Error)’ which is a bit strange for me, as I’m not doing any file operations that I’m aware of.
I’ve turned off Kemal logging via (which I hope is right)
Kemal.config.logging = false
Log.setup(:error) # Only show actual errors
Here’s the full error dump. I’d be grateful for any pointers to help me understand what I should do to pinpoint this.
Unable to load dwarf information: Error opening file with mode 'r': '/Users/michael/Documents/tests/myapp': Too many open files (File::Error)
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::unwind:Array(Pointer(Void))'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack#initialize:Array(Pointer(Void))'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::new:Exception::CallStack'
from /Users/michael/Documents/tests/myapp in 'raise<File::Error+>:NoReturn'
from /Users/michael/Documents/tests/myapp in 'Crystal::System::File::open<String, String, File::Permissions, Nil>:Tuple(Int32, Bool)'
from /Users/michael/Documents/tests/myapp in 'File::new<String, String, File::Permissions, Nil, Nil, Nil>:File'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::read_dwarf_sections:Nil'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::load_debug_info_impl:Nil'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::load_debug_info:Nil'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::decode_line_number<UInt64>:Tuple(String, Int32, Int32)'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::decode_backtrace_frame<Pointer(Void), Bool>:(String | Nil)'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack#decode_backtrace:Array(String)'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack#printable_backtrace:Array(String)'
from /Users/michael/Documents/tests/myapp in 'Exception+@Exception#backtrace?:(Array(String) | Nil)'
from /Users/michael/Documents/tests/myapp in 'Exception+@Exception#inspect_with_backtrace<IO::Memory>:Nil'
from /Users/michael/Documents/tests/myapp in 'Crystal::buffered_message:exception:backtrace<String, Exception+, Nil>:IO::Memory'
from /Users/michael/Documents/tests/myapp in 'Crystal::print_buffered:exception:to<String, Exception+, IO::FileDescriptor>:Nil'
from /Users/michael/Documents/tests/myapp in 'Fiber#run:Nil'
from /Users/michael/Documents/tests/myapp in '~proc2Proc(Fiber, Nil)@/opt/homebrew/Cellar/crystal/1.17.1/share/crystal/src/fiber.cr:105'
Unhandled exception in spawn: Error opening file with mode 'r': '/etc/localtime': Too many open files (File::Error)
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::unwind:Array(Pointer(Void))'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack#initialize:Array(Pointer(Void))'
from /Users/michael/Documents/tests/myapp in 'Exception::CallStack::new:Exception::CallStack'
from /Users/michael/Documents/tests/myapp in 'raise<File::Error+>:NoReturn'
from /Users/michael/Documents/tests/myapp in 'Crystal::System::File::%
BTW I’m using a Deque as I want to limit the number of captured JSON messages into the struct to 1000, and at capacity, I’m unshifting and then pushing the new one in - is that the best way to have this kind of buffer?
Thanks.