Segfault while bootstraping from ruby

Help & Support
1

I’ve been trying to bootstrap the compiler from ruby using GitHub - crystal-lang/bootstrap-script: Automated script to bootstrap the crystal compiler from source, with the goal of updating the script to the latest versions & using containers, but can’t get past the second stage:

==> Bootstrapping Crystal (001/160)
patching file bin/crystal
patching file bootstrap/crystal/compiler.cr
patching file bootstrap/crystal/program.cr
patching file lib/crystal/compiler.rb
                           user     system      total        real
parse                  0.000000   0.000000   0.000000 (  0.000615)
normalize              2.910000   0.120000   3.040000 (  3.030746)
type inference:       26.740000   1.190000  27.930000 ( 27.953474)
fix empty types        0.350000   0.000000   0.350000 (  0.352627)
afert type inference   0.730000   0.020000   0.750000 (  0.739854)
codegen-llvm          17.450000   0.930000  18.380000 ( 18.401006)
codegen-llc          .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s: Assembler messages:
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8847: Warning: stand-alone `data16' prefix
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8849: Warning: stand-alone `data16' prefix
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.s:8850: Warning: stand-alone `data16' prefix
  0.850000   0.170000  54.040000 ( 54.169867)
codegen-clang        /usr/bin/ld: .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.o: in function `_2A_Crystal_3A__3A_Compiler_23_compile_3C_Crystal_3A__3A_Compiler_3E__3A_Nil':
.crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.bc:(.text+0x11b46): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
/usr/bin/ld: warning: .crystal//home/geopjr/bootstrap-script/buildroot/src/crystal/bootstrap/crystal.cr/main.o: missing .note.GNU-stack section implies executable stack
/usr/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
  0.000000   0.110000   0.570000 (  0.576038)
TOTAL:                49.030000   2.540000 105.060000 (105.224228)
==> Bootstrapping Crystal (002/160)
Normalize: 0.618294 seconds
Type inference: 10.2995 seconds
Codegen (crystal): 5.31784 seconds
Codegen (bitcode): 1.07611 seconds
./bootstrap: line 192: 24294 Segmentation fault      (core dumped) "$input" "${crystal_args[@]}" -o "$output" src/compiler/crystal.cr
Normalize: 0.598149 seconds
Type inference: 10.868 seconds
Codegen (crystal): 5.3217 seconds
Codegen (bitcode): 1.17425 seconds
./bootstrap: line 192: 25957 Segmentation fault      (core dumped) "$input" "${crystal_args[@]}" -o "$output" src/compiler/crystal.cr

Using the compiler itself is a bit more verbose:

1-crystal -e "puts \"foo\""

No such file or directory:
_2A_Exception_23_initialize_3C_Errno_2C__20_String_3E__3A_Array_28_String_29_ + [0]
_2A_Errno_23_initialize_3C_Errno_3E__3A_Array_28_String_29_ + [0]
_2A_Errno_3A__3A_new_3C_Errno_3A_Class_3E__3A_Errno + [0]
_2A_File_3A__3A_expand_path_3C_File_3A_Class_2C__20_String_3E__3A_String + [0]
_2A_Crystal_3A__3A_Program_23_require_from_load_path_3C_Crystal_3A__3A_Program_2C__20_String_3E__3A_Nil_20__7C__20_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Program_23_require_3C_Crystal_3A__3A_Program_2C__20_String_2C__20_Nil_20__7C__20_String_20__7C__20_Crystal_3A__3A_VirtualFile_3E__3A_Nil_20__7C__20_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Normalizer_23_transform_3C_Crystal_3A__3A_Normalizer_2C__20_Crystal_3A__3A_Require_2B__3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_ASTNode_2B__23_transform_3C_Crystal_3A__3A_ASTNode_2B__2C__20_Crystal_3A__3A_Normalizer_3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Normalizer_23_transform_3C_Crystal_3A__3A_Normalizer_2C__20_Crystal_3A__3A_Expressions_2B__3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_ASTNode_2B__23_transform_3C_Crystal_3A__3A_ASTNode_2B__2C__20_Crystal_3A__3A_Normalizer_3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Normalizer_23_normalize_3C_Crystal_3A__3A_Normalizer_2C__20_Crystal_3A__3A_ASTNode_2B__3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Program_23_normalize_3C_Crystal_3A__3A_Program_2C__20_Crystal_3A__3A_ASTNode_2B__3E__3A_Crystal_3A__3A_ASTNode_2B_ + [0]
_2A_Crystal_3A__3A_Compiler_23_compile_3C_Crystal_3A__3A_Compiler_3E__3A_Nil + [0]
__crystal_main + [0]
main + [0]
__libc_init_first + [0]
__libc_start_main + [0]
_start + [0]

More info:

  • if you attempt to run the script, it will fail due to pcl’s download location
-- curl -L -o "$downloads"/pcl-1.12.tar.gz http://xmailserver.org/pcl-1.12.tar.gz
++ curl -L -o "$downloads"/pcl-1.12.tar.gz http://www.xmailserver.org/pcl-1.12.tar.gz
  • LLVM 3 requires python 2 (?) which might have been replaced by 3 on your distro (and removed completely from its repos), you can run the following to install it in the build env (run it when it fails and start again after it finishes):
curl -L -o ./downloads/Python-2.7.3.tgz http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz && cd ./buildroot/src/ && rm -Rf Python-2.7.3.tgz && tar xzf ../../downloads/Python-2.7.3.tgz && cd Python-2.7.3 && ./configure --prefix="$(pwd)"/../../ --enable-shared && make && make install && cd ../../../ && rm -rf buildroot/src/llvm-3.3.src/
  • It probably won’t be able to find librt and you might need to symlink it (not sure if this is the best approach but I didn’t want to modify the stage patches yet): ln -s /usr/lib/x86_64-linux-gnu/librt.so.1 ./buildroot/lib/librt.so
  • The script fails on my machine during linking one of the deps, so all my attempts have been on debian containers and vms, including debian versions before the glibc librt & python change, so I doubt the symlink or python2 is the issue

Any ideas? :person_shrugging:

2

what’s the goal?

3

I mean, you mention updating the script and using the container, but do you have a more specific goal?

4

do you have a more specific goal?

Yes, kind-of… there’s a certain distro in alpha with an unusual stack and while figuring out what would be the best way to package Crystal for it I went down the bootstrapping from Ruby rabbit hole. Fixing the script is not that important for reaching that goal as there are better ways to port it over but it’d be nice to have it as a working alternative.

However, I’m about to give up as even using the packages of the time yields the same result and I don’t think anyone is up to debugging a decade old build:

# git clone https://github.com/crystal-lang/crystal/ -b ruby
FROM ubuntu:precise
WORKDIR /app
COPY . .

RUN cat llvm-3.3.tar.gz | tar xz --strip-components=1 -C /usr
RUN sed -i -re 's/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list
RUN apt-get update && apt-get install -y  zlib1g-dev ruby1.9.3 build-essential git libpcre3-dev libunwind7-dev wget


WORKDIR /app/crystal
RUN gem install bundler -v '< 1.8' --no-ri --no-rdoc
RUN bundle
# RUN bin/crystal -e "puts 1" # Works
RUN bin/crystal -stats bootstrap/crystal.cr
RUN ./crystal -e "puts 1"
# segfault
1 Like