for an upcoming open-source project of mine, I want to supply binaries for all major desktop platforms, but at least Linux and OSX. This should be done through a CI-based process, which I can ideally host myself (e.g. with Drone / Docker).
As cross-compiling Crystal is not that easy as e.g. Golang, I am searching for ways to solve this. For compiling to OSX, the things I came across so far:
Travis supports OSX builds. Is Crystal supported there as well? If yes, can you do OSX/Linux matrix builds in order to have one CI pipeline for everything? Also, I read that there might be long queueing times waiting for a build VM to become available.
Sorry, I don’t have any experience with Mac OS build and tooling.
Just wanted to share that CircleCI actually offers Mac OS builds for open source projects for free. You just need to contact them (https://circleci.com/open-source/). AFAIK that’s what we’re using for the Crystal repo as well (but I’m not familiar with the exact details).
The long queue time in travis was something we experience more than a year ago and we turned off osx travis ci and a couple of months later we introduce the CircleCI workflow to do the automated releases. Travis pipeline was not available then.
I was told by travis employees that they worked to solve the osx queue times and it is more was more an incident. I haven’t used osx travis outside crystal-lang.
What you are trying to achieve is to cross-compile from linux and then do the linking in the target platform using hopefully less time of those more expensive/less available resources?
What you are trying to achieve is to cross-compile from linux and then do the linking in the target platform using hopefully less time of those more expensive/less available resources?
Well, ideally, I can do both the compiling and linking on Linux, so I don’t need an OSX machine. As far as I understand, that might be possible with osxcross. (Disclaimer, I am not an expert at this.)
Doing the linking on the target platform (i.e. OSX) does not help much. I do not own an OSX machine, so I would have to use a SaaS platform for the whole process either way.
I am also looking for an easy way to build static binaries that will work on linux and mac.
Although Go has the easiest cross compilation process, if this is hard or not on the roadmap of Crystal to make it work like it does in Go, perhaps borrow the concept from Rust?
Rust has a nice utility called cross that uses docker behind the scenes to build for multiple targets - will that be a viable option for Crystal?
i tried building statically on alpine, hoping it will work on mac, but so far with no luck. Not even sure it is supposed to work.
That’s not possible to work. Linux binaries can’t run natively on Mac OS. They have different kernels, system calls, executable formats.
cross seems nice enough, though. It provides convenient access to complex tool chains for cross compilation. So this is certainly possible to achieve for many targets.
Running cross-compiled binaries natively is impossible though, so for test execution it relies on QEMU.
Very nice, and thanks for explaining. I was able to build for mac on Travis, so I guess this could be a viable alternative for me until things improve.
I hope that the crystal team will consider it a priority to allow users to easily compile to at least linux and mac targets from their own machine, pain free (as in: don’t talk to me about linkers and llvm… ).
As an addicted Rubyist, the primary reason for me trying Crystal is the ability to create binaries. That is not to say that I do not like some of the other improvements, such as the shards system and the performance boost - but the main reason is binaries.
Oh, I said this is a great idea but I don’t think the core team will spend time and effort on this, specially if this can be done as an external project.
Without meaning to hijack this thread - how do people normally publish their crystal app binaries? I mean, my guess is that there is a standard, acceptable workflow that more or less works automatically.
I mean, how do I reach a situation where my crystal binary GitHub page looks like this (tried posting a link, but was rejected by the forum system):
Or are people usually providing just instructions to "clone the repo and run make install"?
I don’t think there are currently many Crystal applications that provide binaries for multiple platforms. The compiler being an exception. Its build workflow is rather complex, though. So it’s not a great example because most applications can get off easier.
My oq shard has support for Mac via brew and amd64 linux via Snap and a static binary. Snap is pretty good as it supports many linux distros and is able to build binaries for various architectures (however currently crystal only supports amd64).
Thank you both. That’s a little disappointing…
I was under the impression that the key selling point of Crystal over Ruby, is the ability to generate binaries. Of course, I know that performance is a key selling point as well - but not being able to easily build for at least common platforms, limits its use cases, and by extension - its growth.
In simple terms, it means that its target audience are people who like Ruby and want to develop internal tooling that demand higher performance than Ruby. That is an extremely small audience I suspect.
So even though Crystal/Ruby’s syntax is (arguably) much easier and fun to use than Go / Rust, if someone or some organization comes to a decision point where they need to provide a binary for public consumption, they will be forced to go elsewhere.
Needless to say, I mean no disrespect here and it is not even a criticism - its just that Crystal is sort of my dream come true: Write Ruby, compile binaries… only the proverbial “compile binaries rug” is being pulled out from under me…
Anyways, FWIW, and thanks again for all your help.
P.S. I think amd64 and osx will cover a big enough audience that it will be sufficient to start with.
For me the idea of Crystal was a Ruby-like language with type-safety, performance and native binaries (no VM). Cross compiling to other platforms was never in my original plans… mainly because when we started I didn’t know what a mess the computer world is in and how difficult it is to achieve that.
That said, I think it’s just a matter of time and community effort before we get a tool like cross for Crystal.
Another option would be to implement all system calls for all platforms in pure Crystal (or inline assembly). Go does that. It’s just that Go has Google behind it and they can afford that time and effort. Our community could do it too but it’s just a beast of a task.
I build with --static and just drop my binary file into my VPS and it works flawlessly. I am curious about the origin of these binary issues. I’ve seen a few on Gitter these past 2 weeks or so.
Maybe I am treating Crystal really good and I’m being rewarded.
Static linked binaries still only run on the same architecture and operating system (more or less). For everything else you have to cross compile. If you don’t need that, good for you.
Also static linking depends on the linked libraries supporting it / playing nice when linked statically. That can lead to trouble. A known troublemaker is for example is openssl, especially when combined with other libraries that depend on it. Again, if you don’t have issues with that, good for you.
Since my main need from Crystal is binaries, and not performance, I must find a way to have binaries available for “all the people I know” - so mac and some linux flavors.
Until there is a crystal --build-for-all-linuxes, I was trying the following approach, and would love it if someone here more familiar with compilers can tell me if this is a viable approach or is it too naive.
This is what I hope is true:
To build for Alpine, I can use docker and build with --static
To build for OSX, I need to build on OSX, and the binary will work for other people using OSX (they might need to brew install something if they have missing libraries).
To build for modern linuxes, running on modern computers, I can build on Ubuntu, and it will work for some other popular linux variants.
So if the above is not all fantasy, I have devised this approach:
Since I do not have access to osx, and travis does have osx builds, I can use it to build for osx.
I can use the same travis process to build for linux, and to build for alpine using docker.
When travis is done, it copies the built files to a GitHub release.
).