If you do a search on the forums will find some replies from me on that subject but these do not go into details.
Decided to sit and write down all that and tried to explain some concept in the first post of 4 in total, covering the subject of cross-compilation and distribution of Crystal applications:
This one is just the basics while I’m cleaning up past experiments and internal code that I’ve been using over the past 3 years
Let me know what do you think about this approach.
There is, it’s towards the top right to the left of the search button. However not every learning resource is in the official docs. Are plenty of quality third-party blog posts, such as this one, that are hosted on a personal blog site, or something like medium or dev.to.
The official docs has Cross-compilation - Crystal, which covers the high level process of cross compiling. I’m sure contributions to improve this page would be welcomed.
I kind of agree with you @aiac, but I’m not part of Crystal team and I did that on my personal blog as I don’t think the quality of it is worth to be considered an official documentation on the whole cross-compilation approach, specially since is a deep subject and hard to tackle for those that are not familiar with it (or others like me, that give certain things for granted).
At least is an starting point that could be used to enrich the official documents mentioned by @Blacksmoke16.
Nice catch! I will be pushing some other fixes (specially grammar ones, as I self-learned english and make a lot of mistakes and use weird expressions!)
What a great article! I tried cross-compiling crystal binaries before but ended up leaving it aside in the linking phase. Today, I tried again with the guidelines about the zig toolchain and successfully cross-compiled a binary that is going to production soon!
One small shortcut that I ended up using was to use apk to download the necessary packages in a chroot environment instead of manually downloading the libraries. From the official crystal docker image, we have:
Which saves some work searching for the right packages. Also, there is a beta version of zig in alpine’s edge branch if you don’t mind adding an unstable repo to your build image. Overall, though, the article was a great source of information; it was just what I needed to get one of the items on my backlog going!
My only concern when using apk is that the version that will be installed may be different than the one I’m expecting, simply because there was a bump/patch/fix in Alpine and now I was silently upgraded to a newer version.
Something that I didn’t cover here is macOS dependencies, for which I will still require to download them.
Downloading a fixed list of versions for both Alpine Linux and macOS is something I already did in the past, but I’m not pleased with all the manual maintenance to keep things up to date (plus that I wrote that in Ruby and not Crystal, hehe).
For the next part of this series I’m trying to clean that up and bring something more manageable, so you don’t have to manually download anything
Well, it does work when building very simple targets. However, when I tried to build the basic service program from the official website example, I encountered an error:
ld.lld: error: undefined symbol: BIO_meth_new
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: BIO_meth_set_write_ex
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: BIO_meth_set_read_ex
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: BIO_meth_set_ctrl
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: BIO_meth_set_create
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: BIO_meth_set_destroy
>>> referenced by main_module
>>> server.o:(__crystal_main)
ld.lld: error: undefined symbol: SSL_write
>>> referenced by main_module
>>> server.o:(*HTTP::WebSocket::Protocol#send<Slice(UInt8), HTTP::WebSocket::Protocol::Opcode, HTTP::WebSocket::Protocol::Flags, Bool>:Nil)
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket+@IO::Buffered#write_byte<UInt8>:(OpenSSL::SSL::Socket+ | Nil))
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket+@IO::Buffered#write_byte<UInt8>:(OpenSSL::SSL::Socket+ | Nil))
>>> referenced 7 more times
ld.lld: error: undefined symbol: zlibVersion
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 4 more times
ld.lld: error: undefined symbol: deflateInit2_
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 1 more times
ld.lld: error: undefined symbol: crc32
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>> server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 4 more times
ld.lld: error: undefined symbol: zError
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Error::new<LibZ::Error, struct.LibZ::ZStream>:Compress::Deflate::Error)
ld.lld: error: undefined symbol: SSL_get_error
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_read<Slice(UInt8)>:Int32)
ld.lld: error: undefined symbol: ERR_get_error
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>> server.o:(*OpenSSL::Error::new<String>:OpenSSL::Error)
ld.lld: error: undefined symbol: ERR_error_string
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>> server.o:(*OpenSSL::Error::new<String>:OpenSSL::Error)
ld.lld: error: undefined symbol: deflate
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Writer#consume_output<LibZ::Flush>:Nil)
ld.lld: error: undefined symbol: BIO_get_data
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Pointer(UInt8), UInt64, Pointer(UInt64), Int32)@/usr/lib/crystal/core/openssl/bio.cr:28)
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Pointer(UInt8), UInt64, Pointer(UInt64), Int32)@/usr/lib/crystal/core/openssl/bio.cr:42)
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Int32, Int64, Pointer(Void), Int64)@/usr/lib/crystal/core/openssl/bio.cr:51)
ld.lld: error: undefined symbol: inflateInit2_
>>> referenced by main_module
>>> server.o:(*Compress::Gzip::Reader#unbuffered_read<Slice(UInt8)>:Int32)
>>> referenced by main_module
>>> server.o:(*HTTP::Request::from_io:max_request_line_size:max_headers_size<IO+, Int32, Int32>:(HTTP::Request | HTTP::Status | Nil))
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Reader::new:sync_close<IO+, Bool>:Compress::Deflate::Reader)
ld.lld: error: undefined symbol: inflate
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Reader#unbuffered_read<Slice(UInt8)>:Int32)
ld.lld: error: undefined symbol: inflateSetDictionary
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Reader#unbuffered_read<Slice(UInt8)>:Int32)
ld.lld: error: undefined symbol: SSL_read
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_read<Slice(UInt8)>:Int32)
ld.lld: error: undefined symbol: deflateEnd
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Writer#close:Nil)
ld.lld: error: undefined symbol: inflateEnd
>>> referenced by main_module
>>> server.o:(*Compress::Deflate::Reader#unbuffered_close:Nil)
ld.lld: error: undefined symbol: SSL_shutdown
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_close:Nil)
ld.lld: error: undefined symbol: BIO_set_shutdown
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:71)
ld.lld: error: undefined symbol: BIO_set_init
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:71)
ld.lld: error: undefined symbol: BIO_set_data
>>> referenced by main_module
>>> server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:84)
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
ld.lld: error: undefined symbol: SSL_new
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
ld.lld: error: undefined symbol: BIO_new
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
ld.lld: error: undefined symbol: SSL_set_bio
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
ld.lld: error: undefined symbol: SSL_free
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
>>> referenced by main_module
>>> server.o:(~proc11Proc(Pointer(Void), Pointer(Void), Nil)@/usr/lib/crystal/core/gc/boehm.cr:211)
ld.lld: error: undefined symbol: SSL_accept
>>> referenced by main_module
>>> server.o:(*OpenSSL::SSL::Socket::Server#accept:Nil)
ld.lld: error: undefined symbol: SHA1
>>> referenced by main_module
>>> server.o:(*HTTP::WebSocketHandler#call<HTTP::Server::Context>:Nil)
Indeed, for the errors you’re listing, you will need to add OpenSSL and Zlib, and a few others, depending on what you’re using.
For reference, here is a long list of all the packages I used in the past:
gc-dev
gmp-dev
libevent-static
libevent-dev
openssl-libs-static
openssl-dev
pcre-dev
pcre2-dev
sqlite-static
sqlite-dev
yaml-static
yaml-dev
zlib-static
zlib-dev
You’re welcome to explore and add the needed dependencies for your test
I mentioned in previous comment, while I would like to ensure reproducibility of the build (being able to get the exact same dependencies), is hard to maintain and keep up to date the entire list.