Milestone: Running an Offline Kemal App with DNF on Fedora Linux! 🎉

Community
1

Hey everyone!

We wanted to share an exciting milestone from our work on the Fedora Crystal SIG (crystal-in-fedora):

Today, we successfully ran a Kemal web application entirely offline inside a network-isolated container (--net=none), after installing Kemal directly as a system library with dnf!

No git clone, no network calls, no shard.yml, no shard.lock, no local lib/ directory, and no vendored tarballs. Just:

# dnf install crystal crystal-kemal

And in a single script (app.cr):

require "kemal"

get "/" do
  "Hello from Fedora Crystal (100% offline)!"
end

Kemal.run

Running crystal run app.cr immediately compiled, resolved all transitive dependencies (radix, exception_page, backtracer), and served HTTP requests completely offline.

How Did We Do It?

While Fedora’s packaging guidelines permit standalone leaf applications to vendor dependencies (similar to Go and Rust using a vendor.tar.xz), distributing reusable shard libraries as unbundled system packages has historically been a challenge. Distributing compiled .so binaries doesn’t work for Crystal because of compile-time macros, generics monomorphization, and whole-program optimizations.

We solved this through three simple mechanisms:

  1. Native System Shard Path: When building the Fedora crystal compiler RPM, we injected /usr/share/crystal/shards into CRYSTAL_CONFIG_PATH:
    CRYSTAL_PATH="lib:/usr/share/crystal/shards:/usr/share/crystal/src"
  2. Shard-as-RPM: We package reusable shard libraries as uncompiled source trees into versioned directories (e.g. /usr/share/crystal/shards/kemal-1.14.0/). This leaves the full AST available to the compiler so macros, generics, and LLVM inlining work natively at build time.
  3. The LLVM Primary Symlink Model: To allow single scripts to do require "kemal" while still letting multiple library versions coexist on the system without RPM package collisions:
    • The primary crystal-kemal package provides an unversioned symlink:
      /usr/share/crystal/shards/kemal -> kemal-1.14.0
    • Compatibility packages (like crystal-kemal1.13) live in their own versioned directory without the unversioned symlink.
    • For complex projects with a shard.lock, our companion tool shards-link reads the lockfile and creates local symlinks in ./lib/ pointing directly to system packages—again, completely offline, reproducible, and without touching Git state.

Multithreading & Fibers

As a bonus during our verification, Crystal 1.21.1’s new Execution Contexts concurrency model and multi-threaded fibers ran effortlessly in the same offline container. Resizing the default execution context or creating worker pools scaled smoothly across system threads:

Observed threads: DEFAULT-0, DEFAULT-1, DEFAULT-2, DEFAULT-3

Concurrent compute fibers executed in parallel across all worker threads with zero hitches.

What Does This Mean for the Ecosystem?

  1. True Air-Gapped Development: You can now develop Crystal applications in high-security, offline environments (finance, telecom, defense, offline laptops) using standard system packages without needing internet access or manual vendoring.
  2. First-Class Distro Shards: Reusable Crystal shard libraries can be installed and managed cleanly via distribution packages, receiving system security updates and CVE tracking.
  3. Frictionless Scripting: Anyone can install a library with dnf and immediately write a one-off script using require "<shard>" without initializing a whole shard workspace.

Come Join the Fedora Crystal SIG!

We are actively building out the Crystal packaging ecosystem in Fedora and EPEL, testing compiler builds on Rawhide and Fedora 44+, and packaging key community shards.

If you love Crystal and want to help with packaging, testing, or shaping how Crystal integrates with Linux distributions, we would love to have you with us!

Cheers,
Rénich Bon Ćirić & Gemini (AI pair programmer collaborating with Rénich)

2

Tracking upstream security advisories (CVEs), coordinating mass/compiler rebuilds, and assisting package maintainers across the distribution.

Will additional resources be given to look for these CVEs as part of this deeper Crystal ↔ Fedora integration?

IMO, supply-chain attacks are an underreported issue, especially in air-gapped development. So I’m glad to see Crystal (or really, Fedora) getting ahead of this.

3

IMHO, more eyes on the code are the resources we need. Especially with the Fedora community. The other day, a Red Hat employee/contributor told me he had been hearing about Crystal lately and that he was looking forward to it being in Fedora.

Fedora is a gateway to a vibrant ecosystem with lots of professionals participating in high collaboration environments and enterprise. If they take a look at Crystal and it brings things to the table they like, it will spread swiftly and widely.

Ruby was famous for a long while in Fedora. The infra team used it some. Even, though, Crystal is not that much like Ruby anymore, it held to that spirit and, in the end, it’s a point of entry to our community.

All my respect goes to the other distros. I do not intend to say what they are and aren’t. I was just acknowledging what Fedora is in my view.

4

All my respect goes to the other distros. I do not intend to say what they are and aren’t. I was just acknowledging what Fedora is in my view.

Having Crystal be more integrated into Fedora’s system installer would almost certainly raise its visibility, as it’s one of the most popular distros.

I’m a long-time Fedora user myself, and I often work in air-gapped systems. So this is right up my alley. I was just curious if incorporating this SIG means there’s a certain amount of security scanning that has to happen before Fedora would allow users to just install shards directly via dnf?

Either way, I’m really excited about this!

5

Hi @mikebelanger,

It definitely changes the equation. Instead of pulling arbitrary Git repositories, shard RPMs are built strictly offline, cryptographically signed, and maintained through Fedora’s centralized review and CVE update pipeline.

On top of that, packaging tools like ameba and Konstantin Dairatchi’s flaw allows us to run automated static security audits directly during package builds. For air-gapped setups, having signed; offline RPMs in an internal mirror provides a reliable chain of custody that raw Git dependencies cannot offer.