AST Normalisation inside def's

Help & Support
1

Consider this source file:

def method
  a, b = 4, 5
  "str"
end

a, b = 4, 5
puts method

The corresponding AST would have two MultiAssign nodes, which should be expanded. Consider running this code through syntactic and semantic stages of the compiler:

require "compiler/crystal/**"

src = Crystal::Compiler::Source.new("filename", <<-CODE)
  def method
    a, b = 4, 5
    "str"
  end

  a, b = 4, 5
  puts method
CODE

compiler = Crystal::Compiler.new
compiler.no_codegen = true
compiler.no_cleanup = false
result = compiler.compile(src, "file_out")
puts result.node

It produces the following output:

<Stuff from prelude>

def method
  a, b = 4, 5
  "str"
end
__temp_305 = 4
__temp_306 = 5
a = __temp_305
b = __temp_306
puts(method)

I am sure method is called. Also, since I used Crystal::Compiler#compile and not Crystal::Compiler#top_level_semantic, I would expect all Visitors and Transformers to run on the whole code.

Am I missing something that only happens at codegen stage? Or am I somehow getting an untransformed ASTNode from Compiler#compile?

What’s really confusing is that if I didn’t set compiler.no_codegen to true, the next operation it’d do is the code generation. It uses Crystal::CodeGenVisitor, which has the following method:

def visit(node : ExpandableNode)
  raise "BUG: #{node} at #{node.location} should have been expanded"
end

But this is not raised when compiling the initial source program normally. (MultiAssign is a subclass of ExpandableNode)

2

Method instantiations are clones of a def that are expanded and typed separately. You will need to check the def_instances property of Program.

2 Likes
3

Does that mean that to get fully transformed methods, types, etc. all information is stored in the Program? In which case, is there any reason why the compile method’s result also includes the AST node?

4

We use it to test the type of things in specs.

5

Sorry for the brevity, I was on my phone.

In specs we do things like assert_type("1 + 2") { int32 } and to do that we run a semantic pass on the node that results from the parsed source and then check its type. We don’t usually check the types of method instantiations in specs (most of our specs are black-box specs so that we can change the implementation without having to rewrite/delete specs).

That said, we use the Program#semantic method for that, not compile. I don’t know why the compile method returns the parsed node, I guess is that so you have the parsed source as a node, otherwise you lose it.

6

@asterite btw, is it normal that DefInstanceContainer#def_instances does not contain methods that take a block? If so, is there a way to get a set of instance methods that do yield?

I know there is ModuleType#defs, but the method bodies aren’t normalised.

7

Yes, it’s normal, because these methods are inlined so the body is always re evaluated.

Side note: to create a transpiler you don’t need semantic analysis, you can just work at the syntax level. So I’m not sure why you are working with def instances and type instantiations.