`a Dockerfile + one line sh docker command` for build specified version's crystal static binary only

Help & Support
1

I am curious if there is a solution for build release binary myself, the reason i want to do this is because i want to enable some preview feature, so have to compile from source.

we can assume following context:

  1. I have crystal git code base locally, and switch to specified tag. e.g. 1.4.1
  2. I have a Dockerfile as template, maybe edited from this Dockerfile
  3. what i want is, only build the crystal static binary one file for X86_84 arch, i consider use distribution-scripts is a little overkill, in fact, it quite slow because network issue, and need install many dependency, i have to change to use mirrors from China to speed things up, and rebuild many duplicate files which i always can download from offical git release repo.

So, any idea? offical distribution-scripts Makefile a little complicated, if someone can help, will appreciated.

2

@zw963 I’m not sure I understand what you’re looking for. I can see that you want to build the crystal compiler yourself. I think you want to do it in a docker container, right?

Can you say more about your goal?

3

e.g. when i run official crystal binary like this:

 ╰─ $ crystal i
Crystal was compiled without interpreter support

For this case, sure i want to compile a new crystal binary only for support interactive crystal, but, i don’t need build everything, so i just hope we can use a more easy way to build crystal static binary only in alpine.

4

Here is a simplified Dockerfile that I use to build Crystal:

# syntax = docker/dockerfile:1.3

# ---
# 1. Use Alpine as base
FROM alpine:3.16 AS base

# ---
# 2. Upgrade system and installed dependencies for security patches
RUN --mount=type=cache,target=/var/cache/apk \
    set -eux; \
    apk upgrade

# ---
# 3. Install common dependencies needed to compile Crystal programs
RUN --mount=type=cache,target=/var/cache/apk \
    set -eux; \
    apk add \
        curl \
        gc-dev \
        gcc \
        git \
        libevent-static \
        musl-dev \
        openssl-dev \
        openssl-libs-static \
        pcre-dev \
        sqlite-static \
        tzdata \
        yaml-static \
        zlib-dev \
        zlib-static \
    ;

# ---
FROM base AS builder

# ---
# 4. Install compiler toolchain and utilities needed to compile Crystal itself
#
# Eg. Alpine's Crystal & Shards
RUN --mount=type=cache,target=/var/cache/apk \
    set -eux; \
    apk add \
        crystal \
        g++ \
        libffi-dev \
        libxml2-dev \
        llvm13-dev \
        llvm13-static \
        make \
        shards \
    ;

Which can be used in Crystal code itself to build the compiler:

$ docker build -t crystal-builder .

$ docker run -it --rm -v $(pwd):/code --workdir /code --user $(id -u):$(id -g) crystal-builder sh -i

/code $ make crystal interpreter=1 static=1 stats=1 
Using /usr/bin/llvm-config [version= 13.0.1]
CRYSTAL_CONFIG_BUILD_COMMIT="7d462845b" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1654787957" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build -D strict_multi_assign --stats --static  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Parse:                             00:00:00.000039454 (   0.69MB)
Semantic (top level):              00:00:00.616233861 ( 134.28MB)
Semantic (new):                    00:00:00.003694679 ( 134.28MB)
Semantic (type declarations):      00:00:00.077105866 ( 142.28MB)
Semantic (abstract def check):     00:00:00.029946789 ( 158.28MB)
Semantic (ivars initializers):     00:00:10.868351978 (1630.21MB)
Semantic (cvars initializers):     00:00:00.014069405 (1630.21MB)
Semantic (main):                   00:00:02.786140894 (1838.21MB)
Semantic (cleanup):                00:00:00.001178661 (1838.21MB)
Semantic (recursive struct check): 00:00:00.003093591 (1838.21MB)
Codegen (crystal):                 00:00:10.150887260 (2382.21MB)
Codegen (bc+obj):                  00:00:15.000596618 (2382.21MB)
Codegen (linking):                 00:00:04.264331394 (2382.21MB)

Macro runs:
 - /code/src/ecr/process.cr: reused previous compilation (00:00:00.004851999)

Codegen (bc+obj):
 - no previous .o files were reused

However, note that by using static linking, you cannot use libffi for dynamic loading:

(in host)

❯ ./bin/crystal i
Using compiled compiler at .build/crystal
cannot find -levent (Dynamic loading not supported)
Linker arguments: -levent
Search path: /usr/lib/x86_64-linux-gnu/libfakeroot:/usr/local/lib:/lib32:/usr/lib32:/usr/local/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/pipewire-0.3/jack/:/lib64:/usr/lib64:/lib:/usr/lib (Crystal::Loader::LoadError)
...

A recommendation will be to install the dependencies covered in the wiki on the host to compile Crystal and then you could use the interpreter.

Hope that helps.

Cheers.

2 Likes
5

Ooh, TIL --mount-type=cache!

1 Like
6

Thank you very much!!

7

I compile crystal successful with interactive REPL enabled on my local laptop!

use make crystal interpreter=1 stats=1 

 ╰─ $ cry version
Crystal 1.5.0-dev [0123976d8] (2022-06-09)

LLVM: 13.0.1
Default target: x86_64-pc-linux-gnu

 ╭─ 00:44  zw963 ⮀ ~/Crystal/share ⮀ ➦ ruby-3.1.0 
 ╰─ $ cry version
Crystal 1.5.0-dev [0123976d8] (2022-06-09)

LLVM: 13.0.1
Default target: x86_64-pc-linux-gnu

 ╭─ 00:44  zw963 ⮀ ~/Crystal/share ⮀ ➦ ruby-3.1.0 
 ╰─ $ cry i
icr:1:0> puts 100  
100
=> nil
icr:2:0> 

 ╰─ $ ldd ~/Crystal/bin/crystal
        linux-vdso.so.1 (0x00007ffd8c1e8000)
        libLLVM-13.so => /usr/lib/libLLVM-13.so (0x00007f98e658f000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f98e6358000)
        libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f98e62e1000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f98e61fa000)
        libevent-2.1.so.7 => /usr/lib/libevent-2.1.so.7 (0x00007f98e61a1000)
        libffi.so.8 => /usr/lib/libffi.so.8 (0x00007f98e6195000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f98e6173000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f98e5f67000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f98eec16000)
        libedit.so.0 => /usr/lib/libedit.so.0 (0x00007f98e5f2c000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00007f98e5f12000)
        libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007f98e5e9e000)
        libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f98e5d14000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f98e5d0d000)
        libicuuc.so.71 => /usr/lib/libicuuc.so.71 (0x00007f98e5b0e000)
        liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f98e5ae4000)
        libicudata.so.71 => /usr/lib/libicudata.so.71 (0x00007f98e3ddf000)
3 Likes
8

That requires DOCKER_BUILDKIT=1 exported as an environment variable.

9

TIL, adding a lone semicolon after the package listing in apk add. Good idea! (Simplifies automatic sorting)