Running Crystal on Termux

HertzDevil · September 22, 2021, 12:47am

I managed to get Crystal working on Termux. This means Crystal can be run on most Android devices, but it is not quite the same thing as running it on an ADB shell. To test it yourself:

Clone this branch;
Cross-compile Crystal with make clean crystal && bin/crystal build --cross-compile --target aarch64-unknown-linux-android24 src/compiler/crystal.cr -Dwithout_playground. (I don’t know if the target triple always ends with android24.)
Copy the created crystal.o file to Termux, somewhere under the home directory.
Ensure you have all of Crystal’s development dependencies installed on Termux.
On Termux run make deps && cc crystal.o -o .build/crystal -rdynamic src/llvm/ext/llvm_ext.o `llvm-config --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm -lgc -lpthread -levent -lrt -ldl -lffi.
Now bin/crystal should work. Back it up, then try building a second generation of the compiler with make crystal. That should work too, including the playground.

All compiler specs pass. Most standard library specs also pass, but some limitations prevent them from getting all green:

Android’s directory structure is different from many Linux systems, e.g. it lacks /usr.
Hard links are not supported. Symbolic links are only supported under Termux’s own directory.
Bionic’s implementation of Iconv supports only a very small subset of custom encodings: ASCII, US-ASCII, UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE, and WCHAR_T. Others like ISO-8859-1 and UCS-2LE will fail if any IO uses them.

Also the following weird error shows up when running spec/std/openssl/ssl/server_spec.cr:

Failed to raise an exception: -484097664
[0x5cbf7baf28] *Exception::CallStack::print_backtrace:Nil +100
[0x5cbf79d214] __crystal_raise +52
[0x5cbf86b140] *Socket+ +140
[0x5cbf86afc0] *Socket+ +200
[0x5cbf7ad52c] ~procProc(Pointer(LibCrypto::Bio), Pointer(UInt8), UInt64, Pointer(UInt64), Int32) +560
[0x70e2feb2d0] ???

Tried to raise:: Error reading socket: Connection reset by peer (IO::Error)
  from src/io/evented.cr:61:9 in 'unbuffered_read'
  from src/io/buffered.cr:80:16 in 'read'
  from src/openssl/bio.cr:46:13 in '->'
  from ???

drhuffman12 · September 22, 2021, 10:12pm

Awesome work! If we could get the ssl server working too, that could totally fit my needs for some projects I had in mind!

drhuffman12 · September 22, 2021, 11:50pm

@HertzDevil , What am I missing?

I cloned the repo, made sure that I had the dependencies installed as noted at All required libraries · crystal-lang/crystal Wiki · GitHub, i.e.:

bdwgc git clone https://github.com/ivmai/bdwgc.git... (I tried both the regular and the upstream patch variants.)
Ubuntu sudo apt-get install \... (but switched to llvm* and lld* 12).

Then, I tried:

make clean crystal && bin/crystal build --cross-compile --target aarch64-unknown-linux-android24 src/compiler/crystal.cr -Dwithout_playground

… but I am getting:

Using /usr/bin/llvm-config-12 [version=12.0.0]
rm -rf .build
rm -rf ./docs
rm -rf src/llvm/ext/llvm_ext.o
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/lib/llvm-12/include -std=c++14   -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="edef02bb1" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1631703912" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Using compiled compiler at .build/crystal
Showing last frame. Use --error-trace for full trace.

In src/exception/call_stack/libunwind.cr:1:1

 1 | require "c/dlfcn"
     ^
Error: can't find file 'c/dlfcn'

If you're trying to require a shard:
- Did you remember to run `shards install`?
- Did you make sure you're running the compiler in the same directory as your shard.yml?

I’m using Linux Mint 20.2 Mate with Crystal 1.1.0 (compiled w/ LLVM 10) and LLVM 12 installed:

$ cat /etc/issue
Linux Mint 20.2 Uma \n \l

$ crystal -v
Crystal 1.1.0 [af095d72d] (2021-07-14)

LLVM: 10.0.1
Default target: x86_64-unknown-linux-gnu

$ llvm-config --version
12.0.0

drhuffman12 · September 23, 2021, 12:50am

I think I’m kinda closer, but I’m still getting the same .. 1 | require "c/dlfcn" .. error. Suggestions? Here’s more info for where I’m at:

I tried to build the official Crystal Compiler using the notes in Install From sources - The Crystal Programming Language and got some errors. Then, I realized that I might need to update my installed Crystal version. (My asdf crystal was maxing out at 1.1.0.) So I switched to the official Crystal 1.1.1 and tried to compile the official Crystal Compiler. That seems to be compiling. (I’m still waiting for tests to complete. )

Meanwhile, I re-tried your instructions with my clone of your repo. Again, I got as far as:

make clean crystal && bin/crystal build --cross-compile --target aarch64-unknown-linux-android24 src/compiler/crystal.cr -Dwithout_playground

I am getting the same .. 1 | require "c/dlfcn" .. error with your repo that I was in my previous post. So, splitting those commands, I get:

make clean crystal: no errors
bin/crystal build --cross-compile --target aarch64-unknown-linux-android24 src/compiler/crystal.cr -Dwithout_playground: This instantly takes me to the ..1 | require "c/dlfcn".. error.

I think c.dlfcn is trying to use some code from glibc. I have:

$ sudo apt list *libc-bin
Listing... Done
libc-bin/now 2.31-0ubuntu9.3 amd64 [installed,local]
libc-bin/focal-updates 2.31-0ubuntu9.2 i386

I even wrapped it in a minimal dockerized alpine container using crystallang/crystal:nightly-alpine-build, but got the same error.

I do see require "c/dlfcn" in the official Crystal Compiler at crystal/libunwind.cr at edef02bb102f969773337e248c4057c96daf4d9a · crystal-lang/crystal · GitHub .

(Also, The official Crystal Compiler finished building and all testes passed. I haven’t actually gotten the official Crystal Compiler to build and run tests locally, so I’m happy for that much progress. )

HertzDevil · September 23, 2021, 11:36am

Did you check out the test/termux branch before doing make clean crystal?

drhuffman12 · September 24, 2021, 5:10am

Yay; thanks! Switching to the test/termux branch got me past the Error: can't find file 'c/dlfcn' issue.

As for Ensure you have all of Crystal’s development dependencies installed on Termux. and subsequent steps, I’ll save that for later, maybe later this weekend. (Some of the dependency package names seem to not be available. Maybe they’re available by alternate names. I’ll have to look them up.)

beta-ziliani · September 24, 2021, 1:44pm

The error you see in server_spec.cr sounds too much to a bug we introduced (but then fixed). Is this on the HEAD of master?

drhuffman12 · September 25, 2021, 6:14am

I got a little further and I’m stuck at one of next steps (cc crystal ...).

See gist a crystal_dependencies_for_termux · GitHub

straight-shoota · September 25, 2021, 7:54am

You need to remove llvm_ext.o and build a new one for your system (make deps).

kefahi · September 26, 2021, 4:49am

Hello,
I have a simpler solution that doesn’t require compilation and works perfectly out of the box.

I have been using Termux with Crystal on my Galaxy Tab s7+ as my primary development machine for the past two months.

First I installed Alpine

pkg update
pkg upgrade
pkg install proot-distro
 
proot-distro install alpine
proot-distro login

Then within Alpine installed Crystal and updated it to the latest version

apk add build-base libc-dev zlib-dev openssl-dev readline-dev gmp-dev yaml-dev libxml2-dev crystal shards
apk add libsodium libsodium-dev libsodium-static # This is extra step specific to my use of libsoduim with crystal
apk upgrade crystal shards \
--repository=http://dl-cdn.alpinelinux.org/alpine/edge/community \
--repository=http://dl-cdn.alpinelinux.org/alpine/edge/main
crystal -v

I hope someone would find this useful.

leapofazzam · December 16, 2021, 8:32am

Got this when rebuilding crystal to run specs

crystal on  test/termux
❯ make crystal
Using /data/data/com.termux/files/usr/bin/llvm-config [version=13.0.0]
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/data/data/com.termux/files/usr/include -std=c++14   -fno-exceptions -fno-rtti -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
CRYSTAL_CONFIG_BUILD_COMMIT="135a809" CRYSTAL_CONFIG_PATH='$ORIGIN/../share/crystal/src' SOURCE_DATE_EPOCH="1638624162" CRYSTAL_CONFIG_LIBRARY_PATH='$ORIGIN/../lib/crystal' ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
Module validation failed: Intrinsic name not mangled correctly for type arguments! Should be: llvm.powi.f64.i32
double (double, i32)* @llvm.powi.f64
 (Exception)
  from /root/crystal/src/llvm/module.cr:100:9 in 'verify'
  from /root/crystal/src/compiler/crystal/codegen/codegen.cr:369:9 in 'finish'
  from /root/crystal/src/compiler/crystal/codegen/codegen.cr:71:7 in 'codegen'
  from /root/crystal/src/compiler/crystal/codegen/codegen.cr:67:5 in 'codegen:debug:single_module'
  from /root/crystal/src/compiler/crystal/progress_tracker.cr:22:7 in 'codegen'
  from /root/crystal/src/compiler/crystal/compiler.cr:173:16 in 'compile'
  from /root/crystal/src/compiler/crystal/command.cr:296:3 in 'compile'
  from /root/crystal/src/compiler/crystal/command.cr:294:5 in 'compile'
  from /root/crystal/src/compiler/crystal/command.cr:187:5 in 'build'
  from /root/crystal/src/compiler/crystal/command.cr:73:7 in 'run'
  from /root/crystal/src/compiler/crystal/command.cr:49:5 in 'run'
  from /root/crystal/src/compiler/crystal/command.cr:48:3 in 'run'
  from /root/crystal/src/compiler/crystal.cr:11:1 in '__crystal_main'
  from /root/crystal/src/crystal/main.cr:110:5 in 'main_user_code'
  from /root/crystal/src/crystal/main.cr:96:7 in 'main'
  from /root/crystal/src/crystal/main.cr:119:3 in 'main'
  from /apex/com.android.runtime/lib64/bionic/libc.so in '__libc_init'
Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues
make: *** [Makefile:180: .build/crystal] Error 1

ItsIgnacioPortal · July 17, 2022, 5:24am

I’m getting this error when running cc:

$ cc crystal.o -o .build/crystal -rdynamic src/llvm/ext/llvm_ext.o `llvm-config --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lpcre -lm -lgc -lpthread -levent -lrt -ldl
ld.lld: error: undefined symbol: ffi_type_pointer
>>> referenced by type.cr:60 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:60)
>>>               crystal.o:(*Crystal::FFI::Type::pointer:Crystal::FFI::Type)
>>> referenced by type.cr:60 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:60)
>>>               crystal.o:(*Crystal::FFI::Type::pointer:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_void
>>> referenced by type.cr:16 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:16)
>>>               crystal.o:(*Crystal::FFI::Type::void:Crystal::FFI::Type)
>>> referenced by type.cr:16 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:16)
>>>               crystal.o:(*Crystal::FFI::Type::void:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_sint8
>>> referenced by type.cr:24 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:24)
>>>               crystal.o:(*Crystal::FFI::Type::sint8:Crystal::FFI::Type)
>>> referenced by type.cr:24 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:24)
>>>               crystal.o:(*Crystal::FFI::Type::sint8:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_uint8
>>> referenced by type.cr:20 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:20)
>>>               crystal.o:(*Crystal::FFI::Type::uint8:Crystal::FFI::Type)
>>> referenced by type.cr:20 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:20)
>>>               crystal.o:(*Crystal::FFI::Type::uint8:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_sint16
>>> referenced by type.cr:32 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:32)
>>>               crystal.o:(*Crystal::FFI::Type::sint16:Crystal::FFI::Type)
>>> referenced by type.cr:32 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:32)
>>>               crystal.o:(*Crystal::FFI::Type::sint16:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_uint16
>>> referenced by type.cr:28 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:28)
>>>               crystal.o:(*Crystal::FFI::Type::uint16:Crystal::FFI::Type)
>>> referenced by type.cr:28 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:28)
>>>               crystal.o:(*Crystal::FFI::Type::uint16:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_sint32
>>> referenced by type.cr:40 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:40)
>>>               crystal.o:(*Crystal::FFI::Type::sint32:Crystal::FFI::Type)
>>> referenced by type.cr:40 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:40)
>>>               crystal.o:(*Crystal::FFI::Type::sint32:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_uint32
>>> referenced by type.cr:36 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:36)
>>>               crystal.o:(*Crystal::FFI::Type::uint32:Crystal::FFI::Type)
>>> referenced by type.cr:36 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:36)
>>>               crystal.o:(*Crystal::FFI::Type::uint32:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_sint64
>>> referenced by type.cr:48 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:48)
>>>               crystal.o:(*Crystal::FFI::Type::sint64:Crystal::FFI::Type)
>>> referenced by type.cr:48 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:48)
>>>               crystal.o:(*Crystal::FFI::Type::sint64:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_uint64
>>> referenced by type.cr:44 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:44)
>>>               crystal.o:(*Crystal::FFI::Type::uint64:Crystal::FFI::Type)
>>> referenced by type.cr:44 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:44)
>>>               crystal.o:(*Crystal::FFI::Type::uint64:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_float
>>> referenced by type.cr:52 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:52)
>>>               crystal.o:(*Crystal::FFI::Type::float:Crystal::FFI::Type)
>>> referenced by type.cr:52 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:52)
>>>               crystal.o:(*Crystal::FFI::Type::float:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_type_double
>>> referenced by type.cr:56 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:56)
>>>               crystal.o:(*Crystal::FFI::Type::double:Crystal::FFI::Type)
>>> referenced by type.cr:56 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/type.cr:56)
>>>               crystal.o:(*Crystal::FFI::Type::double:Crystal::FFI::Type)

ld.lld: error: undefined symbol: ffi_prep_cif
>>> referenced by call_interface.cr:13 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/call_interface.cr:13)
>>>               crystal.o:(*Crystal::FFI::CallInterface::new<Crystal::FFI::Type, Array(Crystal::FFI::Type)>:Crystal::FFI::CallInterface)

ld.lld: error: undefined symbol: ffi_prep_cif_var
>>> referenced by call_interface.cr:36 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/call_interface.cr:36)
>>>               crystal.o:(*Crystal::FFI::CallInterface::variadic:fixed_args<Crystal::FFI::Type, Array(Crystal::FFI::Type), Int32>:Crystal::FFI::CallInterface)

ld.lld: error: undefined symbol: ffi_call
>>> referenced by call_interface.cr:59 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/call_interface.cr:59)
>>>               crystal.o:(*Crystal::FFI::CallInterface#call<Pointer(Void), Pointer(Pointer(Void)), Pointer(Void)>:Nil)

ld.lld: error: undefined symbol: ffi_closure_alloc
>>> referenced by closure.cr:6 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/closure.cr:6)
>>>               crystal.o:(*Crystal::FFI::Closure#initialize<Crystal::FFI::CallInterface, Proc(Pointer(Crystal::LibFFI::Cif), Pointer(Void), Pointer(Pointer(Void)), Pointer(Void), Nil), Pointer(Void)>:Nil)

ld.lld: error: undefined symbol: ffi_prep_closure_loc
>>> referenced by closure.cr:11 (/mnt/c/Users/REDACTED/Desktop/crystal/src/compiler/crystal/ffi/closure.cr:11)
>>>               crystal.o:(*Crystal::FFI::Closure#initialize<Crystal::FFI::CallInterface, Proc(Pointer(Crystal::LibFFI::Cif), Pointer(Void), Pointer(Pointer(Void)), Pointer(Void), Nil), Pointer(Void)>:Nil)
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)

I have no idea why. I haven’t modified the source code at all, and AFAIK those type definitions are already in lib_ffi.cr

Does anyone know what could be causing this?

Also, to get to this stage, I had to run:

chmod +x -R src/llvm/ext/

Because I was getting an “access denied” error when running make deps

And the .build folder didn’t exist for me, so I had to create it manually:

mkdir .build

EDIT: Running with -v gives me this extra output:

clang version 14.0.6
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin
 "/data/data/com.termux/files/usr/bin/ld.lld" --sysroot=/data/data/com.termux/files -pie -export-dynamic -EL --fix-cortex-a53-843419 --warn-shared-textrel -z now -z relro -z max-page-size=4096 --hash-style=gnu --enable-new-dtags -rpath=/data/data/com.termux/files/usr/lib --eh-frame-hdr -m aarch64linux -export-dynamic -dynamic-linker /system/bin/linker64 -o .build/crystal /data/data/com.termux/files/usr/lib/crtbegin_dynamic.o -L/data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -L/system/lib64 crystal.o src/llvm/ext/llvm_ext.o -lLLVM-14 -lc++_shared -lpcre -lm -lgc -lpthread -levent -lrt -ldl /data/data/com.termux/files/usr/lib/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -ldl -lc /data/data/com.termux/files/usr/lib/clang/14.0.6/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -ldl /data/data/com.termux/files/usr/lib/crtend_android.o

asterite · July 17, 2022, 11:48am

You need to install the development libraries for libffi.

Topic		Replies	Views
Hello World from Android ARM64 Crystal Contrib	2	685	January 7, 2023
Cross-compiling Crystal to aarch64 Help & Support	2	637	May 7, 2019
Crystal on Void Linux with Musl	13	1202	January 30, 2020
Cross compiling for second tier Crystal Contrib	4	338	October 20, 2022
Cross-compiling for Raspberry Pi Help & Support	8	947	July 29, 2022

Running Crystal on Termux

Related Topics