How can I use Crystal compiler as a library?

Help & Support
1

Hello.

I am interested in learning more about the Crystal language.

I learned that the Crystal language creates executables via LLVM-IR. And I learned that the following commands work.

Crystal → assembly

crystal build --emit asm --prelude empty test.cr

Crystal → LLVM IR

crystal build --emit llvm-ir --prelude empty test.cr

LLVM IR → assembly

llc --relocation-model=pic test.ll

assembly → executable

clang test.s

Adding libraries to link

clang test.s -levent -lgc -lm

llvm .ll → .bc

llvm-as test.ll
lli test.bc

Then, I learned that there is such a thing as a Lexer or Parser.

require "./src/compiler/crystal/syntax"

str = ARGF.gets_to_end
a = Crystal::Lexer.new(str)
loop do
  t = a.next_token
  puts [t.type,
        t.value,
        t.number_kind,
        t.line_number,
        t.column_number,
        t.filename,
        t.delimiter_state.kind,
        t.delimiter_state.nest,
        t.delimiter_state.end,
        t.delimiter_state.open_count,
        t.delimiter_state.heredoc_indent,
        t.delimiter_state.allow_escapes,
        t.macro_state.whitespace,
        t.macro_state.nest,
        t.macro_state.control_nest,
        t.macro_state.delimiter_state,
        t.macro_state.beginning_of_line,
        t.macro_state.yields,
        t.macro_state.comment,
        t.macro_state.heredocs,
        t.passed_backslash_newline,
        t.doc_buffer,
        t.raw,
        t.start,
        t.invalid_escape,
        t.location].map { |i| i.to_s }.join("\t")
  break if t.type == Crystal::Token::Kind::EOF
end

require "./src/compiler/crystal/syntax"

str = ARGF.gets_to_end
a = Crystal::Parser.parse(str)

def puts_ast(ast)
  puts "\e[32m# #{ast.class}\e[0m" # green
  puts ast
  puts
end

if a.is_a?(Crystal::Expressions)
  a.expressions.each do |ast|
    puts_ast(ast)
  end
else
  puts_ast(a)
end

I would like to get to know Crystal a little deeper. It will take time, but it is not impossible. My small problem is that the compiler is not provided as a library. This makes it difficult to run experimental code using each class of the compiler. This is necessary to understand how the Crystal compiler works.

How can I learn Crystal well?
Or how did you learn it?

6 Likes
2

With the specs?

1 Like
3

Why is this a problem? The compiler’s source code is distributed with the standard library.
You can put a require "compiler/crystal/requires" (or a scoped-down version of it) in your code and the compiler will be available as a library. You can use that for running experimental code, or production code (ameba for example).

3 Likes
4

Thank you, I would like to read the spec little by little.

5

You are right. I thought that the compiler couldn’t be used as a library, but I was wrong.

# ameba code
require "compiler/crystal/syntax/*"

This code works fine. But when I try to use:

require "compiler/requires"

I get an error:

Showing last frame. Use --error-trace for full trace.

In /usr/lib/crystal/lib/compiler/crystal/tools/doc/generator.cr:1:1

 1 | require "../../../../../lib/markd/src/markd"
     ^
Error: can't find file '../../../../../lib/markd/src/markd' relative to '/usr/lib/crystal/lib/compiler/crystal/tools/doc/generator.cr'

How can I fix this?"

6

I’ve ran into this issue too, created a GitHub issue here Move vendored shards to "src/compiler/vendor/" · Issue #13784 · crystal-lang/crystal · GitHub

3 Likes
7

@nobodywasishere
Oh, this is exactly what I wanted to know. This is a good issue.
I ran shard init and added markd, but as expected require "compiler/requires" did not work.

8

Finally, I’ve managed to use the compiler as a library!

crystal init app duck_egg
cd duck_egg

vi shard.yml

name: duck_egg
version: 0.1.0

authors:
  - kojix2 <2xijok@gmail.com>

targets:
  🥚:
    main: src/duck_egg.cr

dependencies:
  markd:
    github: icyleaf/markd
  reply:
    github: I3oris/reply

crystal: '>= 1.15.1'

license: MIT

vi src/duck_egg.cr

require "compiler/requires"

BIRDS = [
  { "🐔", "cluck!" },
  { "🐓", "cock-a-doodle-doo" },
  { "🦃", "gobble" },
  { "🦆", "quack" },
  { "🦉", "hoot" },
  { "🦜", "squawk" },
  { "🕊", "coo" },
  { "🦢", "honk" },
  { "🦩", "brrrrt" },
  { "🐧", "honk honk" },
  { "🦤", "boop" },
  { "🦕", "Bwooooon!!" },
  { "🦖", "Raaaaawr!!" }
]

bird, sound = BIRDS.sample

compiler = Crystal::Compiler.new
source = Crystal::Compiler::Source.new(bird, %Q(puts "#{bird}  < #{sound}"))
compiler.compile source, bird

shards build

# Check CRYSTAL_PATH
crystal env
# Set env
export CRYSTAL_PATH=lib:/usr/local/bin/../share/crystal/src

bin/🥚

./🦖

:t_rex: < Raaaaawr!!

5 Likes