Airsailer - open source Cloud orchestrator in Crystal

Community
1

Hello Crystal community!

Missed you all during my cancer treatment, and I’m finally cancer-free!

I’m back full-time at my cloud computing startup, and going fully open source (MIT license) for the entire platform.

What matters:

  • LXC (Linux/system containers) for bare-metal performance, no VM overhead (PoC working)
  • elastic bare-metal hardware resources that can be changed without reboot
  • elastic LVM storage with thin-provisioning (manually handled today)
  • routing for local networks + WireGuard for clusters (manually handled today)
  • Homogeneous deployment as the strategy for automatic horizontal scaling and load-balancing (will be on a future release with PaaS automation)
  • automatic proxy + SSL for apps, no need for 1 public IP per app (proxy lib temporarily made in JS because of acme-client lib)

Milestone: IaaS+PaaS with 20% of features that cover 80% of cloud demand.

My initial PoC (working, but incomplete) is at: github.com/admiracloud/admira-containers, but please don’t star it.

I’m refactoring and I will release soon at github.com/airsailer/airsailer. Since I sustain my family only through this work, I expect to release a new production version soon.

Happy to talk to you again. :slightly_smiling_face:

17 Likes
2

Glad to have you back. Keep Crystalling!

1 Like
3

Thanks @sdogruyol! A side note, after drafting with multiple Crystal web frameworks, I chose Kemal to manage its web API and the internal unix socket API for the CLI.

The built-in support for auth and web sockets on Kemal, as well as small details like support for unix sockets, made the choice an easy one.

Congratulations!. Kemal is mature, easy and fast, a combination hard to find.

1 Like
4

Super happy to hear that @paulocoghi :blush:

1 Like
5

First, congrats on being cancer-free!

Second, ok cool - airsailer looks neat! Because this is an abstraction over LXC, would this be comparable to say, Docker or Podman?

1 Like
6

Thanks @mikebelanger!

The tl;dr
It should also replace not only Docker or Podman but, on the most common scenarios, also Kubernetes, OpenShift, Microcloud, OpenStack.

It should cover all phases, from local development to production and scaling, plug-and-play.

The decisions and automations to allow that:

[Install and Usage]

  • single binary installation
  • cli and desktop app for local, cli and dashboard for server nodes
  • no dependency on anything other than just LXC

[IaaS]

  • local containers based on directories (LXC in dir mode), automatic lxc bridge and NAT
  • production servers with thin (elastic) LVM volumes, automatic L3 networks
    • router mode for single owner host
    • ipvlan for multiple groups and owners (multi-tenant)
  • wireguard to connect multiple hosts (and their vlans) that are not in the same rack, when clustering

[PaaS]

  • for each project, simple and single TOML file for both IaC and CaC (both infrastructure as code and configuration as code)

  • for each project, a TOML can prepare everything, both local to production:

    • infra (containers)
    • their configuration
    • scaling

  • configuration automation made from recipes based on:

    • “atomic actions” (like changes in a config file)
    • “composite actions” (a sequence of multiple atomic actions for a specific scenario)

[Services exposure and balancing strategy]

  • all services are born in local network:

    • local services, like databases, remain local (even when clusterized among hosts)
    • public services like a backend API can be exposed to the public

  • Airsailer automatically creates HTTP proxies to the exposed services (automatic SSL)

  • these HTTP proxies can also apply load-balancing and fail-over when multiple instances are deployed, also when deployed in multiple separated hosts

  • internal services that are clusterized, like a MariaDB cluster, (or Minio cluster, etc etc) are load-balanced through TCP stream proxies, with transparent fail-over and linear scalability

7

This is the current draft for the web dashboard, which is the same interface for local development (the desktop app).

airsailer - painel
airsailer - painel2560×1440 45.7 KB

8

Ok, very cool! Quadlets can also configure things in a TOML language, and benefit from the broader OCI ecoysystem. How would airsailer compare to something like Quadlets?

The reason I ask is: I’m in the process of learning Podman now (transitioning from Docker). Overall, I find the learning curve pretty intense, especially when SELinux is in the mix too. But I can’t see any viable alternative to it, besides Docker. And even Docker and docker-compose have much of the same issues.

1 Like
9

Both Docker and Podman provide application containers, which follow the principe of “one service per container”.

This principle creates the common scenario of dozens or hundreds containers to be managed, even when you scale mildly.

Quadlets does for Podman what Composer does for Docker, a declarative way to manage the configuration of services in such app containers. Podman is simpler, based on systemd for service life-cycle, but still locked by the app container mentality.

Airsailer is a full cloud platform, but it aims to be as lean and lightweight as Podman (and Quadlets), by following one opinionated recipe for the full life-cycle of your entire app and infrastructure.

Since it is based on LXC, which means system containers, yes, it can deploy stateless apps, but it also has the freedom to do so in simpler ways, since system containers are powerful as a full OS capable to run multiple services.

What changes? For example in a Laravel or Wordpress app without necessity for high scaling:

[Podman Quadlets]

  • one container for Nginx
  • one container for PHP (Laravel or Wordpress)
  • one container for Redis
  • one container for MySQL

[Airsailer]

  • one container for Nginx + PHP + Redis + MySQL

And what about scaling?

  • before going full throttle in multiple containers, more hardware can be added vertically without reboot, to avoid complexity
  • only when vertical scaling is not enough anymore (on many cases it is, please read The Simple Joys of Scaling Up), then Airsailer automatically transitions to a distributed architecture, transparently
  • Docker/Podman/Kubernetes scales by adding more containers, since the beginning

What about deploy?
Both Quadlets and Airsailer can be easily integrated in a CI/CD, for automatic deploy.

But Airsailer allows you to express both the machines with hardware profile, as well as the applications and the services they need, and it will deploy it to the infrastructure already managed by the same software: Airsailer, also in the cloud, with all the other necessary features, including:

DNS, SSL, CDN, Monitoring, cloud servers (LXC containers), CI/CD, IaC, CaC, etc, everything with just one platform.

What about the infra?
You will be able install Airsailer (MIT license) in your own dedicated servers (or a simple VM) or use Airsailer Cloud (with better ROI than the traditional clouds, like 100x less bandwidth cost, 2x perf for the same price, …)

Summary:
You replace everything: Docker (and Composer), Podman (and Quadlets), Terraform, Ansible, Kubernetes, Cloudflare, Openshift, AWS, GCP, Azure, etc

10

About OCI ecosystem, Airsailer will not rely on it.

Instead, it will rely on recipes applied on pure distro images, not on pre-configured services images.

The trade-offs:

  • first installation will require a little bit more time than an OCI image
  • on the other hand, security is better since all packages (OS and services) are always installed in their updated/most recent version at the installation time
  • less worries about outdated images and security vulnerabilities
  • subsequent installations in the same host will be as fast as OCI images because of local cache
11

Wow - sounds cool! I can’t wait to try it out.

1 Like