# Segfault while bootstraping from ruby

**URL:** <https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073>\
**Category:** Help & Support\
**Created:** [October 10, 2023, 4:56pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073 "2023-10-10T16:56:36Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![GeopJr](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/geopjr/32/1051_2.png) [@GeopJr](https://forum.crystal-lang.org/u/GeopJr)\
**Post date:** [October 10, 2023, 4:56pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/1 "2023-10-10T16:56:36Z")

</div>

I’ve been trying to bootstrap the compiler from ruby using [GitHub - crystal-lang/bootstrap-script: Automated script to bootstrap the crystal compiler from source](https://github.com/crystal-lang/bootstrap-script), with the goal of updating the script to the latest versions & using containers, but can’t get past the second stage:

```auto
==> Bootstrapping Crystal (001/160)
patching file bin/crystal
patching file bootstrap/crystal/compiler.cr
patching file bootstrap/crystal/program.cr
patching file lib/crystal/compiler.rb
                           user system total real
parse 0.000000 0.000000 0.000000 ( 0.000615)
normalize 2.910000 0.120000 3.040000 ( 3.030746)
type inference: 26.740000 1.190000 27.930000 ( 27.953474)
fix empty types 0.350000 0.000000 0.350000 ( 0.352627)
afert type inference 0.730000 0.020000 0.750000 ( 0.739854)
codegen-llvm 17.450000 0.930000 18.380000 ( 18.401006)
codegen-llc .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s: Assembler messages:
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8847: Warning: stand-alone `data16' prefix
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8849: Warning: stand-alone `data16' prefix
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8850: Warning: stand-alone `data16' prefix
  0.850000 0.170000 54.040000 ( 54.169867)
codegen-clang /usr/bin/ld: .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.o: in function `_2A_Crystal_3A __3A_Compiler_23_compile_3C_Crystal_3A__ 3A_Compiler_3E__3A_Nil':
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.bc:(.text+0x11b46): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
/usr/bin/ld: warning: .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.o: missing .note.GNU-stack section implies executable stack
/usr/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
  0.000000 0.110000 0.570000 ( 0.576038)
TOTAL: 49.030000 2.540000 105.060000 (105.224228)
==> Bootstrapping Crystal (002/160)
Normalize: 0.618294 seconds
Type inference: 10.2995 seconds
Codegen (crystal): 5.31784 seconds
Codegen (bitcode): 1.07611 seconds
./bootstrap: line 192: 24294 Segmentation fault (core dumped) "$input" "${crystal_args[@]}" -o "$output" src/compiler/crystal.cr
Normalize: 0.598149 seconds
Type inference: 10.868 seconds
Codegen (crystal): 5.3217 seconds
Codegen (bitcode): 1.17425 seconds
./bootstrap: line 192: 25957 Segmentation fault (core dumped) "$input" "${crystal_args[@]}" -o "$output" src/compiler/crystal.cr

```

Using the compiler itself is a bit more verbose:

```auto
1-crystal -e "puts \"foo\""

No such file or directory:
_2A_Exception_23_initialize_3C_Errno_2C __20_String_3E__ 3A_Array_28_String_29_ + [0]
_2A_Errno_23_initialize_3C_Errno_3E__3A_Array_28_String_29_ + [0]
_2A_Errno_3A __3A_new_3C_Errno_3A_Class_3E__ 3A_Errno + [0]
_2A_File_3A __3A_expand_path_3C_File_3A_Class_2C__ 20_String_3E__3A_String + [0]
_2A_Crystal_3A __3A_Program_23_require_from_load_path_3C_Crystal_3A__ 3A_Program_2C __20_String_3E__ 3A_Nil_20 __7C__ 20_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Program_23_require_3C_Crystal_3A__ 3A_Program_2C __20_String_2C__ 20_Nil_20 __7C__ 20_String_20 __7C__ 20_Crystal_3A __3A_VirtualFile_3E__ 3A_Nil_20 __7C__ 20_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Normalizer_23_transform_3C_Crystal_3A__ 3A_Normalizer_2C __20_Crystal_3A__ 3A_Require_2B __3E__ 3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_ASTNode_2B__ 23_transform_3C_Crystal_3A __3A_ASTNode_2B__ 2C __20_Crystal_3A__ 3A_Normalizer_3E __3A_Crystal_3A__ 3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Normalizer_23_transform_3C_Crystal_3A__ 3A_Normalizer_2C __20_Crystal_3A__ 3A_Expressions_2B __3E__ 3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_ASTNode_2B__ 23_transform_3C_Crystal_3A __3A_ASTNode_2B__ 2C __20_Crystal_3A__ 3A_Normalizer_3E __3A_Crystal_3A__ 3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Normalizer_23_normalize_3C_Crystal_3A__ 3A_Normalizer_2C __20_Crystal_3A__ 3A_ASTNode_2B __3E__ 3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Program_23_normalize_3C_Crystal_3A__ 3A_Program_2C __20_Crystal_3A__ 3A_ASTNode_2B __3E__ 3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A __3A_Compiler_23_compile_3C_Crystal_3A__ 3A_Compiler_3E__3A_Nil + [0]
__crystal_main + [0]
main + [0]
__libc_init_first + [0]
__libc_start_main + [0]
_start + [0]

```

More info:

- if you attempt to run the script, it will fail due to pcl’s download location

```diff
-- curl -L -o "$downloads"/pcl-1.12.tar.gz http://xmailserver.org/pcl-1.12.tar.gz
++ curl -L -o "$downloads"/pcl-1.12.tar.gz http://www.xmailserver.org/pcl-1.12.tar.gz

```

- LLVM 3 requires python 2 (?) which might have been replaced by 3 on your distro (and removed completely from its repos), you can run the following to install it in the build env (run it when it fails and start again after it finishes):

```bash
curl -L -o ./downloads/Python-2.7.3.tgz http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz && cd ./buildroot/src/ && rm -Rf Python-2.7.3.tgz && tar xzf ../../downloads/Python-2.7.3.tgz && cd Python-2.7.3 && ./configure --prefix="$(pwd)"/../../ --enable-shared && make && make install && cd ../../../ && rm -rf buildroot/src/llvm-3.3.src/

```

- It probably won’t be able to find librt and you might need to symlink it (not sure if this is the best approach but I didn’t want to modify the stage patches yet): `ln -s /usr/lib/x86_64-linux-gnu/librt.so.1 ./buildroot/lib/librt.so`
- The script fails on my machine during linking one of the deps, so all my attempts have been on debian containers and vms, including debian versions before the glibc librt & python change, so I doubt the symlink or python2 is the issue

Any ideas? 🤷

---

<div class="post-metadata">

**Author:** ![beta-ziliani](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/beta-ziliani/32/1821_2.png) [@beta-ziliani](https://forum.crystal-lang.org/u/beta-ziliani)\
**Post date:** [October 11, 2023, 9:22pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/2 "2023-10-11T21:22:30Z")

</div>

what’s the goal?

---

<div class="post-metadata">

**Author:** ![beta-ziliani](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/beta-ziliani/32/1821_2.png) [@beta-ziliani](https://forum.crystal-lang.org/u/beta-ziliani)\
**Post date:** [October 11, 2023, 9:23pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/3 "2023-10-11T21:23:10Z")

</div>

I mean, you mention updating the script and using the container, but do you have a more specific goal?

---

<div class="post-metadata">

**Author:** ![GeopJr](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/geopjr/32/1051_2.png) [@GeopJr](https://forum.crystal-lang.org/u/GeopJr)\
**Post date:** [October 11, 2023, 10:11pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/4 "2023-10-11T22:11:10Z")

</div>

> do you have a more specific goal?

Yes, kind-of… there’s a certain distro in alpha with an unusual stack and while figuring out what would be the best way to package Crystal for it I went down the bootstrapping from Ruby rabbit hole. Fixing the script is not that important for reaching _that_ goal as there are better ways to port it over but it’d be nice to have it as a working alternative.

However, I’m about to give up as even using the packages of the time yields the same result and I don’t think anyone is up to debugging a decade old build:

```Dockerfile
# git clone https://github.com/crystal-lang/crystal/ -b ruby
FROM ubuntu:precise
WORKDIR /app
COPY . .

RUN cat llvm-3.3.tar.gz | tar xz --strip-components=1 -C /usr
RUN sed -i -re 's/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list
RUN apt-get update && apt-get install -y zlib1g-dev ruby1.9.3 build-essential git libpcre3-dev libunwind7-dev wget

WORKDIR /app/crystal
RUN gem install bundler -v '< 1.8' --no-ri --no-rdoc
RUN bundle
# RUN bin/crystal -e "puts 1" # Works
RUN bin/crystal -stats bootstrap/crystal.cr
RUN ./crystal -e "puts 1"
# segfault

```

---

<div class="post-metadata">

**Author:** ![tycode](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/tycode/32/2699_2.png) [@tycode](https://forum.crystal-lang.org/u/tycode)\
**Post date:** [November 18, 2024, 2:54pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/5 "2024-11-18T14:54:18Z")

</div>

Hi, @GeopJr and other readers…

I’ve also tried to get the bootstrap script working and ran into similar issues (on Debian 12 “bookworm”, in my case). Here’s what I did so far:

1. Like you, I saw I needed Python 2. I built Python 2.7.18 from source and made a symlink so that the `python2` command would run this version.
2. I saw a message about needing to install GraphViz. This is a red herring: GraphViz was already installed, and this message is displayed regardless of whether or not it’s installed. So, you don’t need to worry about this message.
3. I had an error about “Cannot find llvm-config”. This was displayed immediately after the GraphViz message (which is why I ended up looking into that). I’m still not sure why this happened, but after a bit of investigating, I ended up just doing `rm -rf buildroot/src/llvm-3.3.src` and re-running the bootstrap script, and it worked that time.
4. I got the error with `librt.so`. Now I can share some useful information about this error: it happens because that library has been removed from modern glibc. It was only needed by `time.linux.cr`, which is easily fixed with the following patch (add this to the end of `stage1.patch`, and note that the empty-looking line before `class Time` needs a single space on it to be a valid patch file):

```diff
diff --git a/std/time.linux.cr b/std/time.linux.cr
index 8aa07734b..dbe094b9a 100644
--- a/std/time.linux.cr
+++ b/std/time.linux.cr
@@ -1,4 +1,4 @@
-lib Librt("rt")
+lib C
   struct TimeSpec
     tv_sec, tv_nsec : Int64
   end
@@ -7,7 +7,7 @@ end
 
 class Time
   def initialize
- Librt.clock_gettime(0, out time)
+ C.clock_gettime(0, out time)
     @seconds = time.tv_sec + time.tv_nsec / 1e9
   end
 end

```

If you search for `clock_gettime`, you might find yourself [here](https://www.man7.org/linux/man-pages/man3/clock_gettime.3.html), which states “Link with _-lrt_ (only for glibc versions before 2.17)”; that’s how I figured to just change `Librt` to `C`.

1. I got an error about `libunwind.so`. I tried the following workaround (I don’t suggest doing this, as it doesn’t seem to work, see below):

```sh
# as root
cd /usr/lib/x86_64-linux-gnu
ln -s libunwind.so.8.0.1 libunwind.so

```

Doing this _does_ allow stage1 to build, but when it tries to actually run it, it gets a segfault - maybe the same segfault that you’re getting…

I did do a little research, which reveals there’s multiple different `libunwind` projects, and maybe we’re linking to the wrong one. My `libunwind.so.8.0.1`, which is part of my Debian 12 system, comes from [“nongnu” libunwind](http://www.nongnu.org/libunwind), whatever “nongnu” means. But there’s an LLVM libunwind as well, as discussed [here](https://github.com/libunwind/libunwind/discussions/665), and I see it’s mentioned [here](https://clang.llvm.org/docs/Toolchain.html#libunwind-llvm) in the LLVM docs. Since Crystal uses LLVM I’m pondering whether I need to somehow get it to link with the LLVM libunwind instead of the “nongnu” one, and whether that’ll resolve the segfault.

But I’m getting very out of my depth here, so that’s where I stopped. If anyone has some pointers for how to make further progress, I’d really appreciate it!

Hopefully related, I’ve also posted [over on the Debian User Forums](https://forums.debian.net/viewtopic.php?t=160984) to try to find out how Debian is compiling Crystal, since I feel sure they must be bootstrapping it somehow.

---

<div class="post-metadata">

**Author:** ![tycode](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/tycode/32/2699_2.png) [@tycode](https://forum.crystal-lang.org/u/tycode)\
**Post date:** [November 24, 2024, 8:39pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/6 "2024-11-24T20:39:25Z")

</div>

Hi all, I’ve made some progress…!

I installed Debian 7, 8, 9, 10 and 11 in virtual machines, in addition to my desktop Debian 12 installation.

I have now gotten past Stage 2 without a segfault, on Debian 9 & 10 only. On Debian 11 & 12 it still segfaults. I didn’t test 7 & 8 to this point since once I found the success on Debian 9 I felt there was no reason to proceed further on older versions at this point. Note that I cancelled the bootstrap process after it got to Stage 3 due to time constraints, so I don’t yet know if there’ll be further trouble later on.

As I suspected, you _don’t_ want to manually create a `libunwind.so` symlink like I described above. There’s a package for it; it was only a matter of finding it.

From a clean Debian install, as well as ensuring that `python2` runs Python 2, you need at least:

```sh
apt-get install \
  automake \
  build-essential \
  cmake \
  git \
  libgc-dev \
  libpcre3-dev \
  libunwind-dev \
  libyaml-dev \
  zlib1g-dev

```

- `libyaml-dev` and `zlib1g-dev` are required in order to get ruby to build in a way that lets you install gems. If either are missing, then ruby will successfully build, but you’ll get an error later when it tries to install gems, which doesn’t at all point you towards the fact that it’s due to these packages not being installed.
- `libpcre3-dev` is what you need for the bootstrap script to work, not `libpcre2-dev`. Confusingly, although `libpcre2-dev` means PCRE 2, `libpcre3-dev` means PCRE 1 and is older. This had me stumped for a while.
- `libunwind-dev` is (unsurprisingly) the package which creates `libunwind.so`. There are various numbered alternatives available as well, depending on the Debian version.
- `cmake` is only required later in the bootstrap process. I haven’t actually gotten to the point where it’s required yet, but figured I may as well list it.
- **EDIT 2025-01-25** : added `libgc-dev`, this is required for some later stages.

One basic thing I want to explore is if substituting one of the other available `libunwind` packages on Debian 11 and/or 12 resolves the segfault there. I suspect it probably won’t, though, in which case I guess the next thing I’ll want to do is check which other system libraries are being picked up, and whether I can replace more of them with older versions compiled from source, and somehow figure out a combination that won’t segfault.

I’ll also let the bootstrap script run further on 9 & 10 when I have more time, and see whether it can get all the way to the end yet or if I have any further trouble beyond stage 3.

---

<div class="post-metadata">

**Author:** ![RX14](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rx14/32/633_2.png) [@RX14](https://forum.crystal-lang.org/u/RX14)\
**Post date:** [November 25, 2024, 5:19pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/7 "2024-11-25T17:19:38Z")

</div>

I spent a while trying to get this to work again a while ago, on arch linux (the same install I wrote the script on…) I’m starting to suspect I may have to make a more substantial container to reproduce the build.

However, the idea was to have something as lightweight as possible to allow the bootstrap to be reproduced in an official debian package. But the set of things that need to be built into the local prefix is so much larger now, and the failures are harder to debug.

---

<div class="post-metadata">

**Author:** ![tycode](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/tycode/32/2699_2.png) [@tycode](https://forum.crystal-lang.org/u/tycode)\
**Post date:** [November 25, 2024, 6:30pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/8 "2024-11-25T18:30:44Z")

</div>

Thanks for the reply!

I’m planning to do some more work on this soon, and happy to help out where I can. I’ve got several changes to the bootstrap script already - once I’ve got further I can certainly make a PR.

---

<div class="post-metadata">

**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\
**Post date:** [November 25, 2024, 8:22pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/9 "2024-11-25T20:22:21Z")

</div>

> [@tycode](#):
>
> Confusingly, although `libpcre2-dev` means PCRE 2, `libpcre3-dev` means PCRE 1 and is older. This had me stumped for a while.

Yeah, that’s a total footgun 🤦‍♂️ Who could come up with such a crooked idea?

---

<div class="post-metadata">

**Author:** ![tycode](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/tycode/32/2699_2.png) [@tycode](https://forum.crystal-lang.org/u/tycode)\
**Post date:** [January 25, 2025, 11:59am UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/10 "2025-01-25T11:59:46Z")

</div>

Right, now that holidays are over and I’ve had another chance to get further with this…

## LLVM 3.9.1

On Debian 10, after additionally doing `apt install libgc-dev`, it now successfully gets to stage 130, but then it tries to build and install LLVM 3.9.1, which exits with an obscure `ld` error.

So I tried just doing the LLVM 3.9.1 step on its own on Debian 12… where it works fine. I didn’t look into this error any further.

## libunwind

Meanwhile, I looked into the `libunwind` versions. The numbered versions `libunwind-N-dev` are _not_ what you want, because those are the LLVM versions. It is only `libunwind-dev` that is relevant here, which is the NonGNU one. So I tried lifting the version of `libunwind.so` from my Debian 10 installation onto my Debian 12 installation (while rather dangerously temporarily moving my systemwide one on Debian 12 out of the way, so it definitely wouldn’t find it by mistake)… unfortunately it turns out that the stage-2 segfault is still there.

So, I conclude the stage-2 segfault has nothing to do with which version of (NonGNU) libunwind is installed.

## gdb

I tried good old `gdb`, but (as expected) that didn’t produce anything useful either, just that it’s crashing while trying to create a thread:

```auto
(gdb) run -stats -o "/home/.../crystal-lang-bootstrap-script/stages/2-crystal" src/compiler/crystal.cr
Starting program: /home/.../crystal-lang-bootstrap-script/stages/1-crystal -stats -o "/home/.../crystal-lang-bootstrap-script/stages/2-crystal" src/compiler/crystal.cr
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Detaching after vfork from child process 38089]
[Detaching after vfork from child process 38091]
[Detaching after vfork from child process 38096]
[Detaching after vfork from child process 38098]
Normalize: 0.164507 seconds
Type inference: 3.15898 seconds
Codegen (crystal): 1.49343 seconds
[Detaching after vfork from child process 38100]
Codegen (bitcode): 0.445166 seconds
[New Thread 0x7ffff563f6c0 (LWP 38102)]
[New Thread 0x7ffff4e3e6c0 (LWP 38103)]
[New Thread 0x7ffff463d6c0 (LWP 38104)]

Thread 2 "1-crystal" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff563f6c0 (LWP 38102)]
0x00005555ec4413b0 in ?? ()
(gdb) bt
#0 0x00005555ec4413b0 in ?? ()
#1 0x00007ffff69c91c4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#2 0x00007ffff6a4985c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) thread apply all bt

Thread 4 (Thread 0x7ffff463d6c0 (LWP 38104) "1-crystal"):
#0 clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:62
#1 0x0000000000000000 in ?? ()

Thread 3 (Thread 0x7ffff4e3e6c0 (LWP 38103) "1-crystal"):
#0 0x00005555ec432430 in ?? ()
#1 0x00007ffff69c91c4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#2 0x00007ffff6a4985c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 2 (Thread 0x7ffff563f6c0 (LWP 38102) "1-crystal"):
#0 0x00005555ec4413b0 in ?? ()
#1 0x00007ffff69c91c4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#2 0x00007ffff6a4985c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 1 (Thread 0x7ffff7ebc740 (LWP 38086) "1-crystal"):
#0 clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:62
#1 0x00007ffff6a498ad in __GI___ clone_internal (cl_args=cl_args@entry=0x7fffffffd170, func=func@entry=0x7ffff69c8ef0 <start_thread>, arg=arg@entry=0x7ffff463d6c0) at ../sysdeps/unix/sysv/linux/clone-internal.c:54
#2 0x00007ffff69c8df0 in create_thread (pd=pd@entry=0x7ffff463d6c0, attr=attr@entry=0x7fffffffd270, stopped_start=stopped_start@entry=0x7fffffffd266, stackaddr=stackaddr@entry=0x7ffff3e3d000, stacksize=<optimized out>, thread_ran=thread_ran@entry=0x7fffffffd267) at ./nptl/pthread_create.c:295
#3 0x00007ffff69c98bd in __pthread_create_2_1 (newthread=<optimized out>, attr=<optimized out>, start_routine=<optimized out>, arg=<optimized out>) at ./nptl/pthread_create.c:831
#4 0x0000555555b6eb1e in _2A_Thread_28_Crystal_3A __3A_Compiler_2C__ 20_Nil_29 __23_initialize_3C_Thread_28_Crystal_3A__ 3A_Compiler_2C __20_Nil_29__ 2C __20_Crystal_3A__ 3A_Compiler_2C __20_Crystal_3A__ 3A_Compiler_20 __2D__ 3E __20_Nil_3E__ 3A_Void ()
#5 0x0000555555b6eab5 in _2A_Thread_28_Crystal_3A __3A_Compiler_2C__ 20_Nil_29 __3A__ 3A_new_3C_Thread_28_Crystal_3A __3A_Compiler_2C__ 20_Nil_29 __3A_Class_2C__ 20_Crystal_3A __3A_Compiler_2C__ 20_Crystal_3A __3A_Compiler_20__ 2D __3E__ 20_Nil_3E __3A_Thread_28_Crystal_3A__ 3A_Compiler_2C__20_Nil_29_ ()
#6 0x000055555556c4c7 in _2A_Crystal_3A __3A_Compiler_23_compile_3C_Crystal_3A__ 3A_Compiler_3E__3A_Nil ()
#7 0x000055555555a767 in __crystal_main ()
#8 0x000055555555b959 in main ()

```

## other libs

stage-1 crystal doesn’t use many system libraries:

```plaintext
objdump -p stages/1-crystal

...

  NEEDED libpcre.so.3
  NEEDED libunwind.so.8
  NEEDED libLLVM-3.3.so
  NEEDED libm.so.6
  NEEDED libc.so.6
  NEEDED ld-linux-x86-64.so.2

...

```

So I’m really struggling to understand what’s causing this segfault and why it breaks between Debian 10 and 11.

## let’s just lift it

One more thing I tried - since I managed to get stages up to 130 working on Debian 10 but not 12, and LLVM 3.9.1 working on Debian 12 but not 10, I rsync’d the stages up to 130 from 10 to 12 and then ran bootstrap from there on 12 to see what’d happen. It successfully built all the stages from 131 up to 162 but then failed on the very last stage (163… but it’s wrongly numbered 162) with the following error:

```plaintext
src/llvm/ext/llvm_ext.cc: In function ‘LLVMBool LLVMExtCreateMCJITCompilerForModule(LLVMOpaqueExecutionEngine **, LLVMModuleRef, LLVMMCJITCompilerOptions*, size_t, LLVMBool, char** )’:
src/llvm/ext/llvm_ext.cc:548:50: error: ‘AttributeList’ has not been declared
  548 | Attrs = Attrs.addAttribute(F.getContext(), AttributeList::FunctionIndex,
      | ^ ~~~~~~~~~~~~
src/llvm/ext/llvm_ext.cc:561:45: error: no matching function for call to ‘unwrap(LLVMCodeModel&, bool&)’
  561 | if (Optional<CodeModel::Model> CM = unwrap(options.CodeModel, JIT))
      | ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~

```

## So, what next?

We’re at the point now, where if we’re happy to build stages up to 130 on Debian 10, and then switch to 12 for the remaining stages, we just need to fix stage 163 and catch up with the last (4 years of?) additional stages.

With that said - it’d be _far_ preferable if we could do everything on Debian 12 (and newer versions moving forward) - but I’m stumped on what’s causing the stage 2 segfault.

Can anyone offer some help with this? @RX14 have you had any luck in the meantime?

---

<div class="post-metadata">

**Author:** ![RX14](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rx14/32/633_2.png) [@RX14](https://forum.crystal-lang.org/u/RX14)\
**Post date:** [January 25, 2025, 12:57pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/11 "2025-01-25T12:57:16Z")

</div>

I unfortunately haven’t had time to look into it, what really should happen is that the whole build happens in a chroot so the external environment has no effect at all… But that would require building a lot more dependencies to get a full toolchain (libc, clang, etc.)

---

<div class="post-metadata">

**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\
**Post date:** [January 25, 2025, 2:40pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/12 "2025-01-25T14:40:15Z")

</div>

> [@tycode](#):
>
> catch up with the last (4 years of?) additional stages.

That should be relatively easy.  
A Crystal 1.0 compiler can build 1.15 as well, so it’s not necessary to go through all intermediate versions. Neither the build instructions nor dependnecies have changed much, except for the update to libpcre2 in Crystal 1.8.

Although I’m wondering if all the intermediate steps in the bootstrap scropt are even necessary. IIRC there would’ve been some forward compatibility, even pre 1.0.

---

<div class="post-metadata">

**Author:** ![tycode](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/tycode/32/2699_2.png) [@tycode](https://forum.crystal-lang.org/u/tycode)\
**Post date:** [January 25, 2025, 4:05pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/13 "2025-01-25T16:05:21Z")

</div>

> [@RX14](#):
>
> I unfortunately haven’t had time to look into it, what really should happen is that the whole build happens in a chroot so the external environment has no effect at all… But that would require building a lot more dependencies to get a full toolchain (libc, clang, etc.)

Right. I might try learning seriously about `chroot`, then - it’s something I’ve read about a lot by now but never actually used myself.

One thing I’m wondering is whether this stage-2 segfault is caused by some UB, which is “just by accident” not causing a problem on old-enough kernels, but suddenly turns into a crash on newer kernels. If that were the case, `chroot` won’t help, since AIUI it won’t affect the kernel.

> [@straight-shoota](#):
>
> That should be relatively easy.  
> A Crystal 1.0 compiler can build 1.15 as well, so it’s not necessary to go through all intermediate versions. Neither the build instructions nor dependnecies have changed much, except for the update to libpcre2 in Crystal 1.8.
> 
> Although I’m wondering if all the intermediate steps in the bootstrap scropt are even necessary. IIRC there would’ve been some forward compatibility, even pre 1.0.

Yes, I was hoping to go through and remove some of the intermediate builds once I’d gotten the “comprehensive” build working. If the builds can be made reproducible, bit-for-bit identical, then it would be trivial to prove that skipping stage X changes nothing, if the output is identical to not skipping stage X. But getting the bootstrap working at all is my first priority here :)

---

<div class="post-metadata">

**Author:** ![RX14](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rx14/32/633_2.png) [@RX14](https://forum.crystal-lang.org/u/RX14)\
**Post date:** [January 25, 2025, 6:08pm UTC](https://forum.crystal-lang.org/t/segfault-while-bootstraping-from-ruby/6073/14 "2025-01-25T18:08:18Z")

</div>

At least pre-0.1.0, I made pains to make the number of build steps as small as possible. After that some could be removed maybe, but prior to 0.5.0 even the previous release couldn’t guarantee to build the next. Things are messier than they seem, and sometimes you build a compiler succesfully with an old version, but that compiler then compiles a compiler which compiles compilers which segfaults, so you can be sent back 3 or 4 steps when figuring out what broke. Thinning out the steps is a much trickier process than you’d expect.