I want check the document for macro supported expression.
e.g. like following for loop, which is valid only in macro.
{% for key, index in [1,2,3] %}
{% end %}
I saw some example like this in macro reference, but what i expected is, to find the API document for all supported macro statement.
Thank you.
The page you referenced is the macro documentation.
Are you missing something particular?
I think heâs referring to some kind of API documentation reference, but for macros, except easier to navigate than whatâs on the API docs site? I know I run into that issue from time to time, and eventually just becomes trial/error.
API is for classes and methods. But for is syntax. The syntax is documented in the language reference.
Where is the document of full supported syntax of macro?
so, this is all macro document, right? i thought this is just a quick reference, there maybe exist more detailed document will explain why exists a new macro only for keyword, and if there are others keyword which i donât know.
The macro syntax is largely the same as the crystal language syntax, with exception to the for loop youâve mentioned. Iâm not sure why the macro language has for while the compiled language doesnât, but doesnât change much in my opinion.
The Macro node in crystal API docs is my goto for detailed documentation on crystal macros:
Page with top level macro methods: Crystal::Macros - Crystal 1.5.0
Reference for the TypeNode type: Crystal::Macros::TypeNode - Crystal 1.5.0
The TypeNode in particular is useful, since at any given time you have an implicit @type macro variable referencing the current TypeNode, so you can use it to inspect certain things for the class youâre in.
Personally, Iâm really hoping that after the crystal interpreter is finished, we can do a full swap of the current macro interpreted language to actual crystal, and finally have parity between the macro and compiled language 
Iâve been bitten more than a few times by the difference between the methods on Array and ArrayLiteral, Hash and HashLiteral, etc. I fully donât expect this to happen, but I can dream of crystal v2, v3, âŚ
The gist of it the macro code needs access to the main program (e.g. so it can add generated code to it), which you couldnât do with the stdlibâs .each method since its in a block. IIRC there was some sort of discussion around possibly changing that in the future or something. But I canât seem to find it atm.
Really good answer, thank you.
Iâve been bitten more than a few times by the difference between the methods on Array and ArrayLiteral, Hash and HashLiteral, etc
Me too, e.g. there is no easy to to convert a ArrayLiteral to a HashLiteral.
e.g. following to_h method not available on macro, i have to use for ... in loop to recreate a new hash.
# in crystal, not work on macro.
[1,2,3].to_h {|k| [k, k] } # {1=>1, 2=>2, 3=>3}
I should also follow up and mention that there is the normal #each method within macro language, it just cannot be used to generate code. E.g.
{% begin %}
{%
arr = [1, 2, 3]
hash = {} of Nil => Nil
arr.each { |v| hash[v] = v }
%}
{{pp hash}} # => {1 => 1, 2 => 2, 3 => 3}
{% end %}
The main issue with having .each inside macros is this.
Letâs say we introduce that. You can do things like this:
{% [1, 2, 3].each do |x| %}
puts {{x}}
{% end %}
I guess that would generate this code:
puts 1
puts 2
puts 3
But what would this mean?
{% [1, 2, 3].select do |x| %}
puts {{x}}
{% end %}
???
So it only makes sense to allow that syntax with .each, maybe .each_with_index? So I donât know if it makes sense to add that syntax just for that, when a dedicated syntax makes it clear that in general it doesnât make sense to have blocks mixed with macro code.
I really consider we should add following important content into our language reference:
for loop youâve mentioned.for while the compiled language doesnât, The gist of it the macro code needs access to the main program (e.g. so it can add generated code to it), which you couldnât do with the stdlibâs .each method since its in a block.Though, language reference is really a confusing name for our crystal-book site, Welcome - Crystal or About this guide - Crystal ? Both of them said titled as language reference, but, one is index page(including API), another is About this guide when click into, and said. This is a formal specification of the Crystal language., anyway, you know what i means.
This usage is really cool.
BTW, Maybe we should add a to_h into ArrayLiteral?
I think this example is only confusing because Itâs bad code. The problem in the example is that youâre taking a block that is intended to be a mapping (âgiven element x, return true if we should keep xâ) and giving it a side effect (âinterpolate âputs {{x}}â into the sourceâ). The same thing is possible in Crystal already:
[1, 2, 3].select do |x|
# This is the wrong way to use #select - but does it mean blocks are bad style?
puts x # Side effect!
end
Itâs hard to reason about what that code is supposed to do, and it gets even worse for a block thatâs called multiple times with runtime-controlled input (for example, a sorting predicate). I use Crystal macros a lot, and the for loops make macros feel like Python, not Crystal.
This is the wrong way to use #select - but does it mean blocks are bad style?
For those who are new to Crystal, it might help to clarify why this is the wrong way to use #select.
Iâm guessing the âwhyâ would have something to do with using #each instead since #each iterates thru a list (and potentially does something with each item); meanwhile, #select is more for returning of subset of filtered items.
Yeah! My apologies for being too brief.
My point is that #select wants you to give it a predicate - a function that takes an object as input, and returns true or false to the question âshould we keep this object?â Selectâs block is designed to ask you questions. Generally speaking, blocks like this are not called in a useful (or even predictable) order - they might call the block multiple times for certain inputs, none for others, et cetera. The calls might even change as more efficient algorithms are implemented!
On the other hand, #each is a tool whoâs purpose is to predictably call the block you provide for a list of values. In fact, this calling behavior (and the order it does it in) are promised to you by the API documentation.
My point here is that, in @asterite 's example, the confusion is caused by putting side effects (puts {{x}}) into a logical predicate. #select does not provide a calling order as part of itâs API contract, so of course you get undefined behaviour from that side effect. The block syntax is not to blame in this case.
Replacing #select with #each, the block makes perfect sense - #each has a well-defined calling order as part of itâs API.
Hope that helps!
If we were to allow the block syntax in macros⌠What would be the output of the select example? Or would it give a compile error any time you use a method that takes a block thatâs not each or each_with_index?
{% [1, 2, 3].each do |x| %}
puts {{x}}
{% end %}
My opinion is this would be equivalent to the following pseudocode:
{%
[1, 2, 3].each do |x|
body = Call.new(nil, "puts", [MacroExpression.new(Var.new("x"))] of ASTNode)
parse(expand_macro(body, x: x))
end
%}
That expression, of course, is just nil, even if it were printed. Instead one should write:
{{ [1, 2, 3].map do |x| }}
puts {{ x }}
{{ end.to_exps }}
where ArrayLiteral#to_exps constructs an Expressions node with the literalâs elements, so that the above expands to:
begin
puts(1)
puts(2)
puts(3)
end
This gives #select a well-defined meaning:
{{ [1, 2, 3].select do |x| }}
{% if x >= 2 %}
""
{% end %}
{{ end }} # => [2, 3]
{% x = [1, 2, 3].select do |x| %}
{% if x >= 2 %}
def foo{{ x }}; end
{% end %}
{% end %}
{% p x %} # => [def foo2; end, def foo3; end]
The core difference between for and ArrayLiteral#each is that the former is a control flow expression that directs macro interpolation, whereas the latter is a regular macro method which shouldnât be able to interpolate nodes into the surrounding context. I donât think the extra complexity associated with the #map snippet above is really worth it.
What about having a build-in marco method like that:
# Expands *code* into the real crystal code
macro_def expand(code : StringLiteral) : Nop
end
And the two rules like this:
Every code inside a macro but outside {% %} is transformed to expand(<code>).
{{ x }} is transformed to {% expand(x.to_s) %}
Then, to have a build-in macro method capture(&), with the following behavior:
expand and append it to a stringe.g:
{% capture do
expand("x = 42")
end %} # generates nothing, returns `x = 42` (Assign)
{% capture { 0 } %} # generates nothing, returns 0 (NumberLiteral)
{% capture { expand("") } %} # generates nothing, returns Nop
{% capture { expand("+=)") } %} # Error: captured expansion should be syntactically valid.
This way, each and map could be implemented like this:
macro_class ArrayLiteral
macro_def each(&)
i = 0
while i < self.size
yield self[i]
end
end
macro_def map(&)
arr = [] of ASTNode
each do |x|
arr << capture do # we capture expansion and parse it if any, else returns element inside the yield
yield x
end
end
arr
end
end
So then:
{% [1, 2, 3].each do |x| %}
puts {{x}}
{% end %}
# => transformed to:
{% [1, 2, 3].each do |x| %}
puts {% expand(x.to_s) %}
{% end %}
# => transformed to:
{% [1, 2, 3].each do |x|
expand("\n puts ")
expand(x.to_s)
expand("\n")
end %}
# generates: " puts 1\n puts 2\n puts 3\n", returns `Nop`
# it doesn't capture expansions, letting them expand to the code.
Then a map would works intuitively:
{% nodes = [1, 2, 3].map do |x| %}
{% if x % 2 == 0 %}
puts {{x}}
{% else %}
puts {{-x}}
{% end %}
{% end %}
# =>
{% nodes = [1, 2, 3].map do |x|
expand("\n ")
if x % 2 == 0
expand("\n puts ")
expand(x.to_s)
expand("\n ")
else
expand("\n puts ")
expand(-x.to_s)
expand("\n ")
end
expand("\n")
end %}
# => generates nothing, returns [puts(-1), puts(2), puts(-3)] (ArrayLiteral)
# it capture expansions, using them.
And a classical map would works as usual, since capture returns the result of the block if no expansion is performed:
{% x = [1, 2, 3].map(&.to_s) %} # returns ["1", "2", "3"] (ArrayLiteral)
Of course expand, capture and marcro defs donât need to realy exist, it could be hard-coded and hidden for the user.
select, and all other classical methods could be implemented similarly.
This end up to be equivalent to what you stated @HertzDevid, with the key difference that we decide which methods capture or let expand.
But I found this could be a good way to solve at last the dilemma of for vs each.
Thinking?
The only problem I see is if one write something like that: (In a world where marcro defs are widespread)
{% [1, 2, 3 ].map do |x|
foo(x)
x.to_s
end %}
and forgot that foo actually expand something in code, he would be unexpectedly surprised that map captures the code instead of x.to_s.
But in an other hand he could expect that, so itâs maybe not a real problem.