You've found a bug in the Crystal compiler

Help & Support
1

Hi,
Got the following error in a 10K lines application:

BUG: {% for i in 0…T.size %}
yield self[{{ i }}]
{% end %} (Crystal::MacroFor) at /usr/lib/crystal/tuple.cr:367:5 should have been expanded (Exception)
from crystal in ‘??’
from crystal in ‘??’
…
from crystal in ‘??’
from crystal in ‘__crystal_main’
from crystal in ‘main’
from /usr/lib/libc.so.6 in ‘??’
from /usr/lib/libc.so.6 in ‘__libc_start_main’
from crystal in ‘_start’
from ???

This seems to be related to a new use of a class from a personal library, a class already used in several places in the application without problem.

Looking at /usr/lib/crystal/tuple.cr:367:5 didn’t help!

Any clue on how and where to investigate?

2

Could you share the code of this class? It seems to be a codegen error so narrowing the cause would help identify the bug.

3

I have made some progress in exploring the problem.

In parallel with my application, I am developing a library for formatting tables in the terminal, using any data source of type Enumerable (see tablo development and GitHub - hutou/tablo: Crystal text table generator).

In my application, I was so far only using arrays of NamedTuple or struct as the source, and it was when using a simple array of floats (or strings) that the bug appeared (an array of integers, characters, or booleans does not generate the bug).

However, outside my app, a simple test with basic arrays like the ones below gives the expected results.

require "tablo"

ar1 = [1, 2, 3]
ar2 = ['a', 'b', 'c']
ar3 = [false, true, false]
ar4 = [1.1, 2.2, 3.3]
ar5 = ["ab", "bc", "cd"]

def tbl(ar)
  puts(Tablo::Table.new(ar) do |t|
    t.add_column("value") { |n| n }
  end)
end

tbl(ar1)
tbl(ar2)
tbl(ar3)
tbl(ar4)
tbl(ar5)

output:

+--------------+
|        value |
+--------------+
|            1 |
|            2 |
|            3 |
+--------------+
+--------------+
| value        |
+--------------+
| a            |
| b            |
| c            |
+--------------+
+--------------+
|     value    |
+--------------+
|     false    |
|     true     |
|     false    |
+--------------+
+--------------+
|        value |
+--------------+
|          1.1 |
|          2.2 |
|          3.3 |
+--------------+
+--------------+
| value        |
+--------------+
| ab           |
| bc           |
| cd           |
+--------------+

It therefore seems that the problem lies within my application, but where (> 10K LOC) ? Which parts or types of code should be analyzed?

Any idea is welcome!

4

The error is being raised here. The code in tuple.cr is effectively just data being processed — the bug is in the compiler itself.

What you’re likely seeing is a different code path being taken in the compiler due to that change in types you mentioned. I had a look at the tablo code to see what the compiler could be tripping over, but nothing’s immediately jumping out at me and I can’t reproduce the bug.

One hunch I have, though, is do you have any custom struct types in this app that include Enumerable(SomeType)? I’m noticing that Tablo::Table receives @sources : Enumerable(T) and I remember some weird behavior years ago in Crystal when combining struct and class types that include Enumerable. I have no idea if that is still a thing (I can’t reproduce that either), but your generalizing on Enumerable reminded me of it.

Apart from that, your Crystal compiler binary seems to have had debugging info stripped from it, so it’s hard to tell how it ended up at that method I linked above. Any chance you’ve got a self-compiled Crystal compiler handy?

5

Got the last 1.13.3 tarball and tried to compile my app.
Still a compiler bug, but with a more verbose output:

BUG: {% for i in 0…T.size %}
yield self[{{ i }}]
{% end %} (Crystal::MacroFor) at /home/hubert/abs/crystal/crystal-1.13.3-1/share/crystal/src/tuple.cr:367:5 should have been expanded (Exception)
from /crystal/src/compiler/crystal/codegen/codegen.cr:2407:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:28:11 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:2386:9 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:2386:9 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:2386:9 in ‘visit’
from /crystal/src/compiler/crystal/semantic/bindings.cr:12:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/semantic/bindings.cr:12:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:2386:9 in ‘codegen_assign’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/ast.cr:81:7 in ‘codegen_fun’
from /crystal/src/compiler/crystal/codegen/fun.cr:60:3 in ‘target_def_fun’
from /crystal/src/compiler/crystal/codegen/call.cr:435:12 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/exception.cr:92:7 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:793:15 in ‘visit’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/enumerable.cr:510:7 in ‘??’
from /crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in ‘accept’
from /crystal/src/compiler/crystal/codegen/codegen.cr:2356:7 in ‘codegen’
from /crystal/src/compiler/crystal/compiler.cr:211:16 in ‘compile’
from /crystal/src/compiler/crystal/command.cr:239:5 in ‘run’
from /crystal/src/compiler/crystal.cr:11:1 in ‘__crystal_main’
from /crystal/src/crystal/main.cr:118:5 in ‘main’
from src/env/__libc_start_main.c:95:2 in ‘libc_start_main_stage2’
Error: you’ve found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: Issues · crystal-lang/crystal · GitHub

Indeed, Enumerable seems concerned.

6

After further investigation, I can see that the bug only occurs when the data source is a simple array of floats or strings, such as [1.1, 2.2, 3.3] or [“a”, “b”, “c”] or a tuple of these same types (simple array of integers, booleans or chars works fine!)
On the other hand, an array of arrays of these same types, such as [[“a”, “b”, “c”]] does not generate the compiler bug.
In the source code of my application, I have also removed the Enumerable type restrictions in function parameters, but this has no effect.
I’ve also modified a class to avoid an include Indexable, also without effect.

My skills don’t allow me to go much further!
Should I declare an issue despite the lack of explanatory data?

7

I’m afraid we can do much without a reproducible example. Can you pinpoint the diff that caused it?

8

I guess this bug will remain for all eternity if you don’t show the code that throws the exception. Nobody will include the module in an application and then keep trying until the error occurs.

9

Ok, I’ll try to identify the origin of the bug using bisect on my application’s git commits.

2 Likes