Cross-compiling Crystal applications - Part 1

If you do a search on the forums will find some replies from me on that subject but these do not go into details.

Decided to sit and write down all that and tried to explain some concept in the first post of 4 in total, covering the subject of cross-compilation and distribution of Crystal applications:

This one is just the basics while I’m cleaning up past experiments and internal code that I’ve been using over the past 3 years :sweat_smile:

Let me know what do you think about this approach.

Cheers.

11 Likes

I hope there will be a doc section like the Zig forum, instead of jumping to a personal website.

There is, it’s towards the top right to the left of the search button. However not every learning resource is in the official docs. Are plenty of quality third-party blog posts, such as this one, that are hosted on a personal blog site, or something like medium or dev.to.

The official docs has Cross-compilation - Crystal, which covers the high level process of cross compiling. I’m sure contributions to improve this page would be welcomed.

2 Likes

I kind of agree with you @aiac, but I’m not part of Crystal team and I did that on my personal blog as I don’t think the quality of it is worth to be considered an official documentation on the whole cross-compilation approach, specially since is a deep subject and hard to tackle for those that are not familiar with it (or others like me, that give certain things for granted).

At least is an starting point that could be used to enrich the official documents mentioned by @Blacksmoke16.

I hope it helps.

Cheers.

2 Likes

Shouldn’t $ docker run -it --rm -u $(id -u):$(id -g) -v .:/app -w /app sh -i be $ docker run -it --rm -u $(id -u):$(id -g) -v .:/app -w /app crystal-xbuild sh -i ?

Nice catch! I will be pushing some other fixes (specially grammar ones, as I self-learned english and make a lot of mistakes and use weird expressions!) :sweat_smile:

Please keep typos and suggestions coming! :heart:

What a great article! I tried cross-compiling crystal binaries before but ended up leaving it aside in the linking phase. Today, I tried again with the guidelines about the zig toolchain and successfully cross-compiled a binary that is going to production soon!

One small shortcut that I ended up using was to use apk to download the necessary packages in a chroot environment instead of manually downloading the libraries. From the official crystal docker image, we have:

export CHROOT=aarch64-root \
    && mkdir -p $CHROOT/etc/apk/ \
    && cp /etc/apk/repositories $CHROOT/etc/apk/ \
    && cp /etc/resolv.conf $CHROOT/etc/ \
    && apk add -p $CHROOT --initdb -U --arch <DESIRED_ARCH> --allow-untrusted gcc gmp-dev libevent-static musl-dev pcre-dev pcre2-dev \
    gc-dev libxml2-dev libxml2-static openssl-dev openssl-libs-static tzdata yaml-static zlib-static xz-static \
    && find "$CHROOT" -name '*.so*' -delete \
    zig cc -target aarch64-linux-musl <TARGET>.o -o <TARGET> -L"$PWD/$CHROOT/lib" -L"$PWD/$CHROOT/usr/lib" [needed libs for taget] 

Which saves some work searching for the right packages. Also, there is a beta version of zig in alpine’s edge branch if you don’t mind adding an unstable repo to your build image. Overall, though, the article was a great source of information; it was just what I needed to get one of the items on my backlog going!

Ah! Nice trick!

My only concern when using apk is that the version that will be installed may be different than the one I’m expecting, simply because there was a bump/patch/fix in Alpine and now I was silently upgraded to a newer version.

Something that I didn’t cover here is macOS dependencies, for which I will still require to download them.

Downloading a fixed list of versions for both Alpine Linux and macOS is something I already did in the past, but I’m not pleased with all the manual maintenance to keep things up to date (plus that I wrote that in Ruby and not Crystal, hehe).

For the next part of this series I’m trying to clean that up and bring something more manageable, so you don’t have to manually download anything :wink:

Cheers!

Well, it does work when building very simple targets. However, when I tried to build the basic service program from the official website example, I encountered an error:

ld.lld: error: undefined symbol: BIO_meth_new
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: BIO_meth_set_write_ex
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: BIO_meth_set_read_ex
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: BIO_meth_set_ctrl
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: BIO_meth_set_create
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: BIO_meth_set_destroy
>>> referenced by main_module
>>>               server.o:(__crystal_main)

ld.lld: error: undefined symbol: SSL_write
>>> referenced by main_module
>>>               server.o:(*HTTP::WebSocket::Protocol#send<Slice(UInt8), HTTP::WebSocket::Protocol::Opcode, HTTP::WebSocket::Protocol::Flags, Bool>:Nil)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket+@IO::Buffered#write_byte<UInt8>:(OpenSSL::SSL::Socket+ | Nil))
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket+@IO::Buffered#write_byte<UInt8>:(OpenSSL::SSL::Socket+ | Nil))
>>> referenced 7 more times

ld.lld: error: undefined symbol: zlibVersion
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 4 more times

ld.lld: error: undefined symbol: deflateInit2_
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 1 more times

ld.lld: error: undefined symbol: crc32
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::Server::Response::Output#unbuffered_write<Slice(UInt8)>:Nil)
>>> referenced by main_module
>>>               server.o:(*HTTP::CompressHandler::CompressIO#write<Slice(UInt8)>:Nil)
>>> referenced 4 more times

ld.lld: error: undefined symbol: zError
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Error::new<LibZ::Error, struct.LibZ::ZStream>:Compress::Deflate::Error)

ld.lld: error: undefined symbol: SSL_get_error
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_read<Slice(UInt8)>:Int32)

ld.lld: error: undefined symbol: ERR_get_error
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::Error::new<String>:OpenSSL::Error)

ld.lld: error: undefined symbol: ERR_error_string
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Error::new<Pointer(Void), Int32, String>:OpenSSL::SSL::Error)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::Error::new<String>:OpenSSL::Error)

ld.lld: error: undefined symbol: deflate
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Writer#consume_output<LibZ::Flush>:Nil)

ld.lld: error: undefined symbol: BIO_get_data
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Pointer(UInt8), UInt64, Pointer(UInt64), Int32)@/usr/lib/crystal/core/openssl/bio.cr:28)
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Pointer(UInt8), UInt64, Pointer(UInt64), Int32)@/usr/lib/crystal/core/openssl/bio.cr:42)
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Int32, Int64, Pointer(Void), Int64)@/usr/lib/crystal/core/openssl/bio.cr:51)

ld.lld: error: undefined symbol: inflateInit2_
>>> referenced by main_module
>>>               server.o:(*Compress::Gzip::Reader#unbuffered_read<Slice(UInt8)>:Int32)
>>> referenced by main_module
>>>               server.o:(*HTTP::Request::from_io:max_request_line_size:max_headers_size<IO+, Int32, Int32>:(HTTP::Request | HTTP::Status | Nil))
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Reader::new:sync_close<IO+, Bool>:Compress::Deflate::Reader)

ld.lld: error: undefined symbol: inflate
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Reader#unbuffered_read<Slice(UInt8)>:Int32)

ld.lld: error: undefined symbol: inflateSetDictionary
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Reader#unbuffered_read<Slice(UInt8)>:Int32)

ld.lld: error: undefined symbol: SSL_read
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_read<Slice(UInt8)>:Int32)

ld.lld: error: undefined symbol: deflateEnd
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Writer#close:Nil)

ld.lld: error: undefined symbol: inflateEnd
>>> referenced by main_module
>>>               server.o:(*Compress::Deflate::Reader#unbuffered_close:Nil)

ld.lld: error: undefined symbol: SSL_shutdown
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket+@OpenSSL::SSL::Socket#unbuffered_close:Nil)

ld.lld: error: undefined symbol: BIO_set_shutdown
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:71)

ld.lld: error: undefined symbol: BIO_set_init
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:71)

ld.lld: error: undefined symbol: BIO_set_data
>>> referenced by main_module
>>>               server.o:(~procProc(Pointer(LibCrypto::Bio), Int32)@/usr/lib/crystal/core/openssl/bio.cr:84)
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))

ld.lld: error: undefined symbol: SSL_new
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))

ld.lld: error: undefined symbol: BIO_new
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))

ld.lld: error: undefined symbol: SSL_set_bio
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))

ld.lld: error: undefined symbol: SSL_free
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Server#accept?:(OpenSSL::SSL::Socket::Server | Nil))
>>> referenced by main_module
>>>               server.o:(~proc11Proc(Pointer(Void), Pointer(Void), Nil)@/usr/lib/crystal/core/gc/boehm.cr:211)

ld.lld: error: undefined symbol: SSL_accept
>>> referenced by main_module
>>>               server.o:(*OpenSSL::SSL::Socket::Server#accept:Nil)

ld.lld: error: undefined symbol: SHA1
>>> referenced by main_module
>>>               server.o:(*HTTP::WebSocketHandler#call<HTTP::Server::Context>:Nil)

Indeed, for the errors you’re listing, you will need to add OpenSSL and Zlib, and a few others, depending on what you’re using.

For reference, here is a long list of all the packages I used in the past:

  • gc-dev
  • gmp-dev
  • libevent-static
  • libevent-dev
  • openssl-libs-static
  • openssl-dev
  • pcre-dev
  • pcre2-dev
  • sqlite-static
  • sqlite-dev
  • yaml-static
  • yaml-dev
  • zlib-static
  • zlib-dev

You’re welcome to explore and add the needed dependencies for your test :blush:

I mentioned in previous comment, while I would like to ensure reproducibility of the build (being able to get the exact same dependencies), is hard to maintain and keep up to date the entire list.

Cheers.

1 Like