# \[solved\] LLVM Problem linking on arm?

**URL:** <https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750>\
**Category:** Help & Support\
**Created:** [February 25, 2020, 11:28am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750 "2020-02-25T11:28:22Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![mavu](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mavu/32/213_2.png) [@mavu](https://forum.crystal-lang.org/u/mavu)\
**Post date:** [February 25, 2020, 11:28am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/1 "2020-02-25T11:28:22Z")

</div>

Hi

I have built crystal.o as usual on my amd64 machine, and transfered the .o to my RaspberryPi.  
When I try to link it there as usual, i get this and I have no idea where this is coming from.

```auto
pi@raspberrypi:~/crystal $ cc 'crystal.o' -o 'crystal' -rdynamic /home/pi/crystal/src/llvm/ext/llvm_ext.o `/usr/bin/llvm-config-6.0 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm /usr/local/lib/libgc.a -lpthread /home/pi/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/home/pi/crystal/lib -L/usr/lib -L/usr/local/lib
crystal.o: In function `normalize_triple':
/home/mavu/repo/crystal/src/llvm.cr:87: undefined reference to `LLVMExtNormalizeTargetTriple'
crystal.o: In function `initialize':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:5: undefined reference to `LLVMExtNewDIBuilder'
crystal.o: In function `create_compile_unit':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:9: undefined reference to `LLVMExtDIBuilderCreateCompileUnit'
crystal.o: In function `create_file':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:25: undefined reference to `LLVMExtDIBuilderCreateFile'
crystal.o: In function `create_basic_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:13: undefined reference to `LLVMExtDIBuilderCreateBasicType'
crystal.o: In function `get_or_create_type_array':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:17: undefined reference to `LLVMExtDIBuilderGetOrCreateTypeArray'
crystal.o: In function `create_subroutine_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:21: undefined reference to `LLVMExtDIBuilderCreateSubroutineType'
crystal.o: In function `create_function':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:34: undefined reference to `LLVMExtDIBuilderCreateFunction'
crystal.o: In function `set_current_debug_location':
/home/mavu/repo/crystal/src/llvm/builder.cr:243: undefined reference to `LLVMExtSetCurrentDebugLocation'
crystal.o: In function `create_lexical_block':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:29: undefined reference to `LLVMExtDIBuilderCreateLexicalBlock'
crystal.o: In function `set_current_debug_location':
/home/mavu/repo/crystal/src/llvm/builder.cr:243: undefined reference to `LLVMExtSetCurrentDebugLocation'
crystal.o: In function `name':
/home/mavu/repo/crystal/src/llvm/basic_block.cr:22: undefined reference to `LLVMExtBasicBlockName'
crystal.o: In function `create_basic_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:13: undefined reference to `LLVMExtDIBuilderCreateBasicType'
crystal.o: In function `create_enumerator':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:59: undefined reference to `LLVMExtDIBuilderCreateEnumerator'
crystal.o: In function `get_or_create_array':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:55: undefined reference to `LLVMExtDIBuilderGetOrCreateArray'
crystal.o: In function `create_enumeration_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:63: undefined reference to `LLVMExtDIBuilderCreateEnumerationType'
crystal.o: In function `create_pointer_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:78: undefined reference to `LLVMExtDIBuilderCreatePointerType'
crystal.o: In function `create_replaceable_composite_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:82: undefined reference to `LLVMExtDIBuilderCreateReplaceableCompositeType'
crystal.o: In function `create_member_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:73: undefined reference to `LLVMExtDIBuilderCreateMemberType'
crystal.o: In function `create_struct_type':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:68: undefined reference to `LLVMExtDIBuilderCreateStructType'
crystal.o: In function `replace_temporary':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:86: undefined reference to `LLVMExtDIBuilderReplaceTemporary'
crystal.o: In function `create_auto_variable':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:39: undefined reference to `LLVMExtDIBuilderCreateAutoVariable'
crystal.o: In function `create_expression':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:47: undefined reference to `LLVMExtDIBuilderCreateExpression'
crystal.o: In function `insert_declare_at_end':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:51: undefined reference to `LLVMExtDIBuilderInsertDeclareAtEnd'
crystal.o: In function `create_parameter_variable':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:43: undefined reference to `LLVMExtDIBuilderCreateParameterVariable'
crystal.o: In function `end':
/home/mavu/repo/crystal/src/llvm/di_builder.cr:90: undefined reference to `LLVMExtDIBuilderFinalize'
crystal.o: In function `write_bitcode_with_summary_to_file':
/home/mavu/repo/crystal/src/llvm/module.cr:62: undefined reference to `LLVMExtWriteBitcodeWithSummaryToFile'
collect2: error: ld returned 1 exit status

```

crystal.o should have been built using llvm-6.0 because that is the only version those 2 platforms currently have in common. (still using raspbian stretch on the PI and LLVM-6 is the latest one in that version)

I say should have been built, because “make crystal” shows which LLVM version it is using to build, but when cross-compiling I don’t see a way to show which LLVM version it is actually using.

to build crystal.o on amd64:

```auto
export LLVM_CONFIG=/usr/bin/llvm-config-6.0
./bin/crystal build src/compiler/crystal.cr --cross-compile --target armv6-unknown-linux-gnueabihf -s -D without_openssl -D without_zlib

```

linker command:

```auto
cc 'crystal.o' -o 'crystal' -rdynamic /home/pi/crystal/src/llvm/ext/llvm_ext.o `/usr/bin/llvm-config-6.0 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm /usr/local/lib/libgc.a -lpthread /home/pi/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/home/pi/crystal/lib -L/usr/lib -L/usr/local/lib

```

Any ideas where those errors could be coming from?  
I guess some LLVM version mismatch? but then how do I specify the correct LLVM versions when building to cross-compile?

edit: Just to be sure, I removed all LLVM versions except 6.0 on my build machine, and that still does not change anything.

---

<div class="post-metadata">

**Author:** ![rogerdpack](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rogerdpack/32/117_2.png) [@rogerdpack](https://forum.crystal-lang.org/u/rogerdpack)\
**Post date:** [February 25, 2020, 3:28pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/2 "2020-02-25T15:28:47Z")

</div>

was this solved? what was the fix if so?

---

<div class="post-metadata">

**Author:** ![mavu](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mavu/32/213_2.png) [@mavu](https://forum.crystal-lang.org/u/mavu)\
**Post date:** [February 25, 2020, 3:34pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/3 "2020-02-25T15:34:28Z")

</div>

Ok, I didn’t find the issue, but I found a way to make it work:

- Download and unpack official release tarball.
- point CRYSTAL\_LIBRARY\_PATH to where you unpacked it.  
`export CRYSTAL_LIBRARY_PATH=/home/mavu/crystal_build/crystal-0.33.0-1/share/crystal/src/`
- checkout crystal sources and build the compiler locally with the downloaded one:  
`PATH=/home/mavu/crystal_build/crystal-0.33.0-1/bin:$PATH make`
- change CRYSTAL\_LIBRARY\_PATH to crystal sources (where you just built the compiler):  
`export CRYSTAL_LIBRARY_PATH=/home/mavu/repo/crystal/src`
- cross-compile for armv6:  
`./bin/crystal build src/compiler/crystal.cr --cross-compile --target armv6-unknown-linux-gnueabihf -s -D without_openssl -D without_zlib`
- copy resulting crystal.o to RaspberryPi
- git clone / checkout same version of the compiler on the PI
- make deps on the PI
- Change the paths and execute the line on the PI the cross-compile command gave you at the end:  
`cc 'crystal.o' -o 'crystal' -rdynamic /home/pi/crystal/src/llvm/ext/llvm_ext.o `/usr/bin/llvm-config-6.0 --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm -lgc -lpthread /home/pi/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/home/pi/crystal/src -L/usr/lib -L/usr/local/lib`

That should give you a working compiler that runs on the raspberryPI.  
I don’t know if it is really neccessary to first build your own compiler on x86-64 and then use that to cross compile.  
It is possible that I did something wrong before I tried this way and it will work without doing that.  
This process worked for me on Crystal versions 0.30.0 -\> 0.33.0 (just tried it on all of those releases)

---

<div class="post-metadata">

**Author:** ![mavu](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mavu/32/213_2.png) [@mavu](https://forum.crystal-lang.org/u/mavu)\
**Post date:** [February 25, 2020, 3:35pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/4 "2020-02-25T15:35:01Z")

</div>

was typing the solution while you posted 🙂

---

<div class="post-metadata">

**Author:** ![mavu](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mavu/32/213_2.png) [@mavu](https://forum.crystal-lang.org/u/mavu)\
**Post date:** [February 25, 2020, 3:47pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/5 "2020-02-25T15:47:29Z")

</div>

ok, this is great.  
I did some more tests and I don’t know why, but now I can cross-compile without building a local crystal-compiler or downloading a tar-release.

I seem to have fixed something along the way, and the only things that come to mind are that I explicitly set the CRYSTAL\_LIBRARY\_PATH to the git-checkout of crystal when cross-compiling, and that I installed libgc-dev (debian buster) somewhere in the middle of all this.

I want to do a “clean” test and see if I can reproduce this on a fresh install and provide a better path to crystal on the PI than this (possibly unnecessarily) way above.

---

<div class="post-metadata">

**Author:** ![rvprasad](https://avatars.discourse-cdn.com/v4/letter/r/c4cdca/32.png) [@rvprasad](https://forum.crystal-lang.org/u/rvprasad)\
**Post date:** [May 14, 2020, 6:44am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/6 "2020-05-14T06:44:16Z")

</div>

Hey,

I just tried your instructions but I couldn’t get it work. Specifically, the resulting compiler fails to execute a simple snippet such as `print("Hello")`. So, can you please confirm if the cross-compiled compiler can execute programs?

FYI, I had jotted down similar steps that had yielded better success on raspbian/buster. Today, I tried revising them for debian/buster Docker image and that led me to your question here. You can find my steps [here](https://github.com/rvprasad/docker-files/tree/master/crystal-for-buster-armhf).

Thanks,

---

<div class="post-metadata">

**Author:** ![jkridner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jkridner/32/1823_2.png) [@jkridner](https://forum.crystal-lang.org/u/jkridner)\
**Post date:** [April 27, 2023, 5:55pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/7 "2023-04-27T17:55:23Z")

</div>

With the current version of Crystal (1.8.1), there is an llvm\_ext.o that is requested for linking against, but is never built. The Makefile will build it, but does not set the cross-compile flags, even if you set ‘target’.

---

<div class="post-metadata">

**Author:** ![jkridner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jkridner/32/1823_2.png) [@jkridner](https://forum.crystal-lang.org/u/jkridner)\
**Post date:** [April 27, 2023, 6:58pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/8 "2023-04-27T18:58:47Z")

</div>

For those that come later, it was _not_ clear to me that what I needed to do was build llvm\_ext.o (via `make deps`) on my _target_ platform. I was able to get `crystal` linked this way.

The interactive mode won’t start, but I’m hoping I have enough of the build mode to get the compiler bootstrapped such that I can publish an aarch64 container where I can build future compiler releases and any updates I might make.

---

<div class="post-metadata">

**Author:** ![luislavena](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/luislavena/32/572_2.png) [@luislavena](https://forum.crystal-lang.org/u/luislavena)\
**Post date:** [April 27, 2023, 7:24pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/9 "2023-04-27T19:24:35Z")

</div>

Hello, if you’re interested in a container image for aarch64, you could take a look to mine: [hydrofoil-crystal](https://github.com/luislavena/hydrofoil-crystal), which is bootstrapped using existing Alpine Linux Crystal version and then cross-compile the compiler to the target platform (both linux/amd64 and linux/arm64 images are provided with the same tag).

I use it for development on Intel and aarch64 when accessing remote, but also for cross-compilation.

Feel free to inspect at the Dockerfile for the process taken.

Hope it helps! 😊

---

<div class="post-metadata">

**Author:** ![jkridner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jkridner/32/1823_2.png) [@jkridner](https://forum.crystal-lang.org/u/jkridner)\
**Post date:** [April 27, 2023, 8:00pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/10 "2023-04-27T20:00:46Z")

</div>

This looks awesome, but how do you run the same Dockerfile on multiple arch at one pass? It seems all the stages would run on the same arch and there is not any qemu invocation.

---

<div class="post-metadata">

**Author:** ![luislavena](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/luislavena/32/572_2.png) [@luislavena](https://forum.crystal-lang.org/u/luislavena)\
**Post date:** [April 27, 2023, 8:25pm UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/11 "2023-04-27T20:25:37Z")

</div>

The trick is using `--platform` on the `FROM` conditions to use the image for the build platform (could be amd64, could be arm64): [hydrofoil-crystal/Dockerfile at main · luislavena/hydrofoil-crystal · GitHub](https://github.com/luislavena/hydrofoil-crystal/blob/main/docker/1.8/Dockerfile#L13)

```plaintext
FROM --platform=$BUILDPLATFORM alpine:3.17.3 AS stage0

```

Then you build using `TARGETARCH`:

```auto
make crystal release=1 static=1 target=$TARGETARCH-alpine-linux-musl | tail -1 | tee .build/crystal.sh; \

```

And last you just do `FROM` without a platform, which will use the one you provided to the `docker build` commands.

This only works with BuildKit powered setups (been available since 2020 or so) and uses a multi-stage dockerfile to copy things around.

I described things while experimenting in [this pull request](https://github.com/luislavena/hydrofoil-crystal/pull/42).

Since emulating arm64 in GitHub runners was painfully slow, I’m using [depot.dev](https://depot.dev/) for docker builders, which spins up two machines: a Graviton2 (arm64) and a Intel to build both aarch64 and x86\_64, respectively, taking like 10 minutes to build both:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/crystal_lang/original/2X/5/5ee3fe719e1948d41e1cf217df77f2a2eacfae44.png)

Cheers!

---

<div class="post-metadata">

**Author:** ![jkridner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jkridner/32/1823_2.png) [@jkridner](https://forum.crystal-lang.org/u/jkridner)\
**Post date:** [April 29, 2023, 2:19am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/12 "2023-04-29T02:19:16Z")

</div>

I’ve got an aarch64 gitlab instance where I’m trying to build it. I’ve got buildkit, but I’m trying to use Debian, rather than Alpine. Just hacking, but I’m getting closer.

> **[test (#10357) · Jobs · Jason Kridner / crystal · GitLab](https://git.beagleboard.org/jkridner/crystal/-/jobs/10357)**
>
> Git repositories for BeagleBoard.org projects

I have a somewhat odd error from my perspective with trying to get interactive mode working where a `stat` reference isn’t resolved. Not sure why it cannot link properly with functions in `libc`!

---

<div class="post-metadata">

**Author:** ![zw963](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/zw963/32/1623_2.png) [@zw963](https://forum.crystal-lang.org/u/zw963)\
**Post date:** [May 3, 2023, 10:08am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/13 "2023-05-03T10:08:54Z")

</div>

Just for you probably not know.

interactive mode not work with static build for linux currently.

---

<div class="post-metadata">

**Author:** ![jkridner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jkridner/32/1823_2.png) [@jkridner](https://forum.crystal-lang.org/u/jkridner)\
**Post date:** [May 7, 2023, 4:54am UTC](https://forum.crystal-lang.org/t/solved-llvm-problem-linking-on-arm/1750/14 "2023-05-07T04:54:09Z")

</div>

But the interpreter/interactive mode should work on ARM, correct? I’m no longer trying to build static… I just want to start with a working ARM build (with interactive mode) on Debian 11.

> **[test\_interpreter (#10542) · Jobs · Jason Kridner / crystal · GitLab](https://git.beagleboard.org/jkridner/crystal/-/jobs/10542)**
>
> Git repositories for BeagleBoard.org projects