Can't compile Crystal 1.2.2 on 32-bit ARM (armv7l) device: undefined reference to `__multi3'

Help & Support
1

Hello everyone. I can’t compile crystal on my armv7l device with Ubuntu 20.04. Here are the steps that I followed:

Some info about my x86_64 device:

thesseyren@x86_64:~$ uname -a
Linux tsslenovo 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
thesseyren@x86_64:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal
thesseyren@x86_64:~$ crystal -v
Crystal 1.2.2 [6529d725a] (2021-11-10)

LLVM: 10.0.0
Default target: x86_64-unknown-linux-gnu

Version list of required libraries on x86_64:

||/ Name                Version                      Arch
+++-===================-============================-============-
ii  automake            1:1.16.1-4ubuntu6            all         
ii  build-essential     12.8ubuntu1.1                amd64       
ii  crystal             1.2.2-1+1.1                  amd64  (from official crystal deb repository)
ii  libbsd-dev:amd64    0.10.0-1                     amd64       
ii  libedit-dev:amd64   3.1-20191231-1               amd64       
ii  libevent-dev        2.1.11-stable-1              amd64       
ii  libgc-dev:amd64     1:7.6.4-0.4ubuntu1           amd64       
ii  libgc1c2:amd64      1:7.6.4-0.4ubuntu1           amd64       
ii  libgmp-dev:amd64    2:6.2.0+dfsg-4               amd64       
ii  libgmpxx4ldbl:amd64 2:6.2.0+dfsg-4               amd64       
ii  libpcre3-dev:amd64  2:8.39-12build1              amd64       
ii  libssl-dev:amd64    1.1.1f-1ubuntu2.9            amd64       
ii  libtool             2.4.6-14                     all         
ii  libxml2-dev:amd64   2.9.10+dfsg-5ubuntu0.20.04.1 amd64       
ii  libyaml-dev:amd64   0.2.2-1                      amd64       
ii  lld-12              1:12.0.0-3ubuntu1~20.04.4    amd64       
ii  llvm-12             1:12.0.0-3ubuntu1~20.04.4    amd64       
ii  llvm-12-dev         1:12.0.0-3ubuntu1~20.04.4    amd64

I did these on x86_64:

thesseyren@x86_64:~$ sudo ln -sf /usr/bin/ld.lld-12 /usr/bin/ld.lld
thesseyren@x86_64:~$ git clone https://github.com/crystal-lang/crystal.git
thesseyren@x86_64:~$ cd crystal/
thesseyren@x86_64:~/crystal$ git checkout 1.2.2
thesseyren@x86_64:~/crystal$ make release=1 progress=1 target=armv7l-unknown-linux-gnueabihf
Using /usr/bin/llvm-config-12 [version=12.0.0]
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-12/include -std=c++14   -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="6529d725a" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1636546449" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build --release --progress --cross-compile --target armv7l-unknown-linux-gnueabihf  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
cc .build/crystal.o -o .build/crystal  -rdynamic -L/usr/bin/../lib/crystal src/llvm/ext/llvm_ext.o `"/usr/bin/llvm-config-12" --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm -lgc -lpthread -levent -lrt -ldl
thesseyren@x86_64:~/crystal$ file .build/crystal.o
.build/crystal.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), with debug_info, not stripped

Cross-compile command printed and object file successfully generated.

Now the exact setup on armv7l device:

thesseyren@armv7l:~$ uname -a
Linux localhost 3.4.5-6164126 #1 SMP PREEMPT Mon Oct 17 15:39:25 KST 2016 armv7l armv7l armv7l GNU/Linux
thesseyren@armv7l:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal
thesseyren@armv7l:~$ llvm-config-12 --host-target
armv7l-unknown-linux-gnueabihf

Version list of required libraries on armv7l:

||/ Name                Version                      Arch
+++-===================-============================-============-
ii  automake            1:1.16.1-4ubuntu6            all         
ii  build-essential     12.8ubuntu1.1                armhf       
ii  libbsd-dev:armhf    0.10.0-1                     armhf       
ii  libedit-dev:armhf   3.1-20191231-1               armhf       
ii  libevent-dev        2.1.11-stable-1              armhf       
ii  libgc-dev:armhf     1:7.6.4-0.4ubuntu1           armhf       
ii  libgc1c2:armhf      1:7.6.4-0.4ubuntu1           armhf       
ii  libgmp-dev:armhf    2:6.2.0+dfsg-4               armhf       
ii  libgmpxx4ldbl:armhf 2:6.2.0+dfsg-4               armhf       
ii  libpcre3-dev:armhf  2:8.39-12build1              armhf       
ii  libssl-dev:armhf    1.1.1f-1ubuntu2.9            armhf       
ii  libtool             2.4.6-14                     all         
ii  libxml2-dev:armhf   2.9.10+dfsg-5ubuntu0.20.04.1 armhf       
ii  libyaml-dev:armhf   0.2.2-1                      armhf       
ii  lld-12              1:12.0.0-3ubuntu1~20.04.4    armhf       
ii  llvm-12             1:12.0.0-3ubuntu1~20.04.4    armhf       
ii  llvm-12-dev         1:12.0.0-3ubuntu1~20.04.4    armhf

I did these on armv7l:

thesseyren@armv7l:~$ sudo ln -sf /usr/bin/ld.lld-12 /usr/bin/ld.lld
thesseyren@armv7l:~$ git clone https://github.com/crystal-lang/crystal.git
thesseyren@armv7l:~$ cd crystal/
thesseyren@armv7l:~/crystal$ git checkout 1.2.2
thesseyren@armv7l:~/crystal$ mkdir .build
thesseyren@armv7l:~/crystal$ make deps
Using /usr/bin/llvm-config-12 [version=12.0.0]
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-12/include -std=c++14   -fno-exceptions -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
thesseyren@armv7l:~/crystal$ file src/llvm/ext/llvm_ext.o
src/llvm/ext/llvm_ext.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped

And transferred program object file from x86_64 to armv7l:

thesseyren@x86_64:~$ scp .build/crystal.o armv7l:crystal/.build

As final step, I run cross compile output:

thesseyren@armv7l:~/crystal$ cc .build/crystal.o -o .build/crystal  -rdynamic -L/usr/bin/../lib/crystal src/llvm/ext/llvm_ext.o `"/usr/bin/llvm-config-12" --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm -lgc -lpthread -levent -lrt -ldl
/usr/bin/ld: .build/crystal.o: in function `__muloti4':
/home/thesseyren/Masaüstü/crystal-arm/crystal/src/crystal/compiler_rt/mul.cr:42: undefined reference to `__multi3'
/usr/bin/ld: .build/crystal.o: in function `//':
/home/thesseyren/Masaüstü/crystal-arm/crystal/src/int.cr:105: undefined reference to `__multi3'
/usr/bin/ld: /home/thesseyren/Masaüstü/crystal-arm/crystal/src/int.cr:105: undefined reference to `__multi3'
/usr/bin/ld: .build/crystal.o: in function `_carrying_mul':
/home/thesseyren/Masaüstü/crystal-arm/crystal/src/crystal/compiler_rt/divmod128.cr:45: undefined reference to `__multi3'
/usr/bin/ld: .build/crystal.o: in function `_carrying_mul_add':
/home/thesseyren/Masaüstü/crystal-arm/crystal/src/crystal/compiler_rt/divmod128.cr:56: undefined reference to `__multi3'
/usr/bin/ld: .build/crystal.o:/home/thesseyren/Masaüstü/crystal-arm/crystal/src/crystal/compiler_rt/divmod128.cr:161: more undefined references to `__multi3' follow
collect2: error: ld returned 1 exit status

What do you think? Is it because old linux kernel version, incompatible llvm version or maybe not recently updated libgc from ubuntu repositories?.. Please reply what do you think, thanks.

I tested this on a different armv7l device and I got the same result:

thesseyren@another-armv7l:~$ uname -a
Linux localhost 3.10.9-8842622 #1 SMP PREEMPT Thu Jul 28 07:22:59 KST 2016 armv7l armv7l armv7l GNU/Linux

I also tested this on a 64-bit aarch64 device and it just worked! :

Linux localhost 4.14.117-perf-gbbfb2f0 #1 SMP PREEMPT Sun Oct 25 23:27:54 CST 2020 aarch64 Android
1 Like
2

I thought 32-bit arch support was removed recently, but I can’t seem to find it in the release notes.

1 Like
3

32-bit arm architectures are expected to work (see Platform Support - Crystal). But we don’t run CI on those platforms, so apparently something broke there and we didn’t notice. The compiler process works fine, the linker is just complaining about a missing symbol.

The unresolved symbol points to a missing compiler-rt library. If you install that on the target machine, your program should link.

We probably should implement this method in Crystal to avoid the dependency on compiler-rt. We already do that for some other functions from that library.

4

I installed compiler-rt (which is provided by llvm-dev on debian) and still have the same issue.

5

Is there something to add to the ld command to link against compiler-rt?

6

You can pass --link-flags to the build command.
But this topic is about cross-compilation. For that you can just edit the linker command directly.

Is your use case the same as the OP? If not, please open a new topic.

7

Same use case, same error when running cc (not ld, sorry).

I installed llvm-dev which provides compiler-rt on debian bullseye, but I got the same error. I am wondering if I have to edit the cc command.

8

The linker command in the OP already includes -lrt, so you shouldn’t need to do anything else.

However, I see that __multi3 might not be available on 32-bit arm platforms. It’s used for 128-bit integer support.

Best bet is probably to use Crystal 1.1.1 for now.

2 Likes
9

I filed a bug report here: Can't build any programs on 32-bit ARM · Issue #11497 · crystal-lang/crystal · GitHub

This does not affect the compiler specifically, but every Crystal program. The issue includes a patch you can apply to your stdlib. Or go with 1.1.1

2 Likes