My question is: How are flags meant to be persistent, at least for host_flags?
What I’m trying to do:
I’ve compiled Crystal 1.17.1 with clang-20 from MacPorts. The MacPorts provided libiconv uses the libiconv function names, rather than iconv. This seems to be already well-handled by the Crystal standard library, but I’m having run-time problems.
My Makefile.local is:
progress = 1
threads = 8
verbose = 1
interpreter = 1
FLAGS = --verbose -Duse_libiconv
LDFLAGS = -rpath /opt/local/lib -rpath /opt/local/libexec/llvm-20/lib
export LDFLAGS
LLVM_CONFIG = /opt/local/bin/llvm-config-mp-20
export LLVM_CONFIG
CC = /opt/local/bin/clang-mp-20
export CC
LD = /opt/local/bin/lld-link-mp-20
export LD
The resulting crystal fails on basic I/O tests that require iconv due to the “use_libiconv” flag not being set when the compiler runs.
Example failure in make test:
1) ARGV accepts UTF-8 command-line arguments
Failure/Error: fail "Compiler command `#{compiler} #{args.join(" ")}` failed with status #{status}.#{"\n" if output}#{output}"
Compiler command `bin/crystal build -o /var/folders/t8/fscs9s514y50wy52_d38qrlc0000gn/T/cr-spec-31c17fe7/kernel/executable_file /var/folders/t8/fscs9s514y50wy52_d38qrlc0000gn/T/cr-spec-31c17fe7/kernel/source_file` failed with status 1.
Using compiled compiler at .build/crystal
ld: Undefined symbols:
_iconv, referenced from:
_*Crystal::Iconv#convert<Pointer(Pointer(UInt8)), Pointer(UInt64), Pointer(Pointer(UInt8)), Pointer(UInt64)>:UInt64 in C-rystal5858I-conv.o0.o
_iconv_close, referenced from:
_*Crystal::Iconv#close:Nil in C-rystal5858I-conv.o0.o
_iconv_open, referenced from:
_*Crystal::Iconv#initialize<String, String, (Symbol | Nil)>:Nil in C-rystal5858I-conv.o0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with exit status 1: /opt/local/bin/clang-mp-20 "${@}" -o /var/folders/t8/fscs9s514y50wy52_d38qrlc0000gn/T/cr-spec-31c17fe7/kernel/executable_file -rdynamic -L/usr/local/bin/../lib/crystal -L/opt/local/lib -lgc -lpthread -liconv
This can be demonstrated more simply from a simple command-line to evaluate the use_libconv flag:
% crystal eval 'puts {{ flag?(:use_libiconv) }}'
ld: Undefined symbols:
_iconv, referenced from:
_*Crystal::Iconv#convert<Pointer(Pointer(UInt8)), Pointer(UInt64), Pointer(Pointer(UInt8)), Pointer(UInt64)>:UInt64 in C-rystal5858I-conv.o0.o
_iconv_close, referenced from:
_*Crystal::Iconv#close:Nil in C-rystal5858I-conv.o0.o
_iconv_open, referenced from:
_*Crystal::Iconv#initialize<String, String, (Symbol | Nil)>:Nil in C-rystal5858I-conv.o0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with exit status 1: cc "${@}" -o /Users/jkridner/.cache/crystal/crystal-run-eval.tmp -rdynamic -L/usr/local/bin/../lib/crystal -L/opt/local/lib -lgc -lpthread -liconv
% crystal eval -Duse_libiconv 'puts {{ flag?(:use_libiconv) }}'
true
% crystal eval -Duse_libiconv 'puts {{ host_flag?(:use_libiconv) }}'
false
So, why doesn’t Crystal remember that I want the compiler to use libiconv? How should I be running the tests?
OK, now that I’ve posed the question, I did go and look at what the MacPorts maintainer did to build crystal in the MacPorts distro. I’m personally not familiar with Portfile syntax or style, but what seems clear is:
- the dependency is in the
libiconvprovided within the distro, - they similarly fixed-up the rpath for
llvm, - they did NOT use the
use_libiconvflag, - they copied
src/lib_c/*-openbsd/c/iconv.crtosrc/lib_c/*-darwin/c/iconv.cr.
Not surprising, the OpenBSD iconv.cr uses the libiconv function names. This hardly seems like a suitable work-around, so how is this supposed to work? Should MacPorts have a new target that is not “-darwin” or should Crystal somehow be remembering use_libiconv?
Finding these downstream hacks has become a real pet peeve for me, so I hope we can clear this up together.
From the perspective of trying to create some more targets for Crystal, it would be just good for me to understand the mindset behind using flags, if they are always considered to be ephemeral, and what should really go into a lib_c target.
For reference, my installed ports are:
% port installed requested
The following ports are currently installed:
binaryen @123_0 (active)
bison @3.8.2_2+universal (active)
boehmgc @8.2.8_1 (active)
cargo @0.89.0_0 (active)
clang-20 @20.1.8_0+analyzer (active)
clang_select @2.4_0 (active)
cmake @3.31.7_0+universal (active)
coreutils @9.5_1 (active)
create-dmg @1.2.2_0 (active)
curl @8.13.0_0+brotli+darwinssl+http2+idn+psl+universal+zstd (active)
curl-ca-bundle @8.13.0_0 (active)
dfu-util @0.11_0 (active)
flex @2.6.4_0+universal (active)
gawk @5.3.2_0+universal (active)
glib2 @2.78.4_2+universal+x11 (active)
go @1.24.5_0 (active)
gptfdisk @1.0.9_1 (active)
harfbuzz @11.2.1_1 (active)
hidapi @0.12.0_0 (active)
ImageMagick @6.9.13-21_0+x11 (active)
libarchive @3.8.1_0+universal (active)
libiconv @1.17_0
llvm-20 @20.1.8_0 (active)
llvm-devel @20240308-6a0618a0_0 (active)
llvm_select @2_1 (active)
md4c @0.5.2_0+universal (active)
nasm @2.16.03_0 (active)
nbd @3.20_0 (active)
ninja @1.12.1_1+universal (active)
nmap @7.97_0+pcre+ssl (active)
opencv4-devel @4.9.0_5 (active)
openssh @10.0p2_2 (active)
pandoc @3.7.0.2_0 (active)
pkgconfig @0.29.2_0 (active)
py-hid @1.0.5_0 (active)
py-jupyter @1.0.0_3 (active)
py-pygraphviz @1.11_0 (active)
py27-tkinter @2.7.18_0 (active)
py27-virtualenv @20.15.1_0 (active)
py313-markupsafe @3.0.2_0 (active)
py313-pygraphviz @1.11_0 (active)
py313-setuptools @80.9.0_0 (active)
python27 @2.7.18_10+lto+optimizations+universal (active)
qemu @10.0.2_0+cocoa+curses+spice+spice_protocol+target_arm+target_i386+target_x86_64+usb+vnc (active)
rust @1.88.0_0 (active)
swig @4.3.1_0 (active)
swig-python @4.3.1_0 (active)
tio @3.9_0 (active)
tree @2.2.1_0 (active)
wasm3 @0.5.0_0 (active)
wget @1.25.0_1+gnutls (active)