Creating a compiler package for Tumbleweed’s main repository (and potentially bootstrapping the compiler)

Crystal Contrib
1

Hi all,

I have been using openSUSE Tumbleweed for a few years now and am maintaining a small number of packages for that distribution. A few months ago I went the (probably) common route from Ruby to Crystal and have been quite happy with it. I thought about getting a Crystal compiler package into Tumbleweed’s main repository, which would have the great benefit of enabling getting packages of other programs written in Crystal into the main repository as well, apart from making the compiler install process easier.
I found the packages at https://build.opensuse.org/project/show/devel:languages:crystal but unfortunately they just extract a prebuilt binary, which, as far as I know, is not acceptable for Tumbleweed. Additionally, even though it is on the same build service and the name looks like it, it is not a devel project for Tumbleweed (which it would need to be to submit a package).

Effectively, this means the following:

  1. Each compiler version package must be built from source.
  2. Either devel:languages:crystal needs to become a devel project for Tumbleweed, or a new project needs to be created. (The naming conflict would need to be solved, potentially together with the Tumbleweed maintainers.)

The first point is especially interesting because, if I understand correctly, up until 1.0, each compiler version needs the previous version to compile. I see 2 solutions for this problem:

  • Bootstrap a recent compiler from the old Ruby implementation.
  • Use a prebuilt binary to compile a recent compiler in the package build process, and base packages for the following versions on that.

How this has to be handled depends on the Tumbleweed maintainers. Unfortunately, I haven’t received an answer through the appropriate Matrix and Discord channels, so I guess I have to try the mailing list next.

For the second point, as per the openSUSE Wiki, it may be as easy as writing an email to a specific mailing list. One requirement is that the factory-maintainers group would become a maintainer of the project. (I don’t see this as an issue, just wanted to point it out in case someone else does.)

I am not sure how both points would affect the package builds for other distributions. DPKG-based distributions could be completely unaffected, but RPM-based ones seem tricky. Their packages could also be built from source, but if that is not realistically possible for even a single one, the source-compiling Tumbleweed packages may need to be separate.
If it needs to be separate, I think a new subproject devel:languages:crystal:Factory would be most fitting, to not bother users of the existing project.

I would be willing and (probably) able to do most if not all of the packaging work (apart from what I have no permissions for). With this post I want to get the thoughts of you Crystal compiler people on that topic in general, and specifically for the following questions:

  1. What do you think about having the Crystal compiler in Tumbleweed’s main repository?
  2. Is there a preference for changing existing package build instructions vs. having a new subproject specifically for Tumbleweed (with the option of building for openSUSE Leap as well)?
  3. If the compiler needs to be bootstrapped, would someone be able to support me a bit in debugging? (Outside of this thread, of course.)
1 Like
2

Hey, thanks for the offer to help out. It would certainly be nice to get a Crystal package into the distro repository.
Happy to help with that.

I don’t have much experience with openSUSE and how package management works there.
We’re using the OBS project to build and distribute packages for several distros, both RPM and DEB based.
So far, it’s only repackaging the statically linked executable. We’ve wanted to upgrade to a full native build process, which would allow dynamic linking which is required for shipping the interpreter. But available expertise in this area is limited.
We’ve also been considering migrating away from OBS because it has some limitation for our use case. Not decided on this, though. For now it’s working fine, although we would like more.

Ping @uncomfyhalomacro who’s been helping out with the build project.

Bootstrapping has come up before for similar purposes of getting Crystal into source-only repositories. Unfortunately, this process is quite a mess. The bootstrap script should be a good starting point, though.

There’s a similar discussion about this in the forum: Segfault while bootstraping from ruby

I recall other discussions about bootrapping packages, but I cannot find a link nor remember where it was.

3

FWIW, bootstrapping has an exception on binaries

Some software (usually related to compilers or cross-compiler environments) cannot be built without the use of a previous toolchain or development environment (open source). If you have a package which meets this criteria, contact the openSUSE Packaging Committee for approval.

Bootstrapping from Ruby is not an option and an overkill for this to be honest.

4

Hi @expeehaa and everyone here. I actually successfully set up crystal with one exception that I didn’t use the bootstrapping process. As @GeopJr said, there are some exceptions so yes I went to that route.

I also ensured that during my experiments is allowing multiple versions of crystal, I also have made it so by hacking the RPM specfile and make it similar to how openSUSE packages Rust. See my home project here and take a look at the Show home:uncomfyhalomacro - openSUSE Build Service to check the crystal packages. If you need help, you can refer to me. It might take a while to get a response from me though.

5

One thing to take note of is I am not an expert in Debian/Apt. In my opinion, although it’s a good idea to have it in the main projects and install crystal from there and not from the devel project, it would hinder the Crystal team from distributing the compiler for multiple distributions. But would gladly help if they change their mind.

6

Thanks for all the replies!

I apparently missed that statement. That’s great, then I won’t bother.

This is very helpful. I’m using it as a basis for my modifications now, thanks.

You mean that each minor release has its own package, right? I would have done that as well. Although I would go a step further and allow installing multiple versions at the same time, similar to how it is handled with the Python packages (unless there is a good reason not to do that, in which case please tell me).

I’m not sure I understand that correctly. Does that mean that you would be in favour of having separate packages specifically for openSUSE:Factory in a project that is not devel:languages:crystal? I somewhat intentionally did not mention in my first post that this would be my preferred option, because it wouldn’t affect existing workflows.
I have not worked with the packaging side for APT yet and have, currently, absolutely no intention of doing so. Either way, I would’ve avoided to touch that part of the package sources.

I will finish the spec file hopefully in the next few days and would then, unless someone objects to it, send an email to the openSUSE packaging mailing list about the package requiring an external binary to build and the devel project proposal.