On the behaviour of Enumerable#reduce(&.+)

Somewhat surprisingly, the following compiles, and returns 1

[1,2].reduce(&.+)

@Blacksmoke16 highlighted that the above is equivalent to

[1,2].reduce { |v| v.+ }

which raises a couple of unrelated questions

  • What is a use case where the shorthand syntax makes sense for Enumerable#reduce? Should [1,2].reduce(&:any_method) even compile?
  • Should the method with no arguments Number#+ even be a thing?
3 Likes

There are also seem to be other inconsistencies with methods Number#+ & Number#- as well. I think these may be due to the fact that these 2 methods can take either 0 or 1 argument (be either a unary or binary operator).

2.+

returns 2 which is expected, as 2.+ is the same as +2. But

2 +

gives the error “no overload matches ‘Int32#+’ with type Nil”, whereas I would have expected “wrong number of arguments”.

And:

[1,2].reduce(&.*)

doesn’t compile with the error “wrong number of arguments”. I would have expected this to compile & return 2, like Ruby does.

It appears that Crystal is attempting to call the method with no arguments. To test:

struct Int32
  def test
    puts "No arg"
    3
  end

  def test(i)
    puts "Arg: #{i}"
    self + i
  end
end

puts 2.test
puts 1.test 4
puts [1,2,3].reduce(&.test)

Gives the output:

No arg
3
Arg: 4
5
No arg
No arg
3

Yep, the method in the reduce block is being called with no arguments. Bug?

How do you get this error? crystal eval '2 +' errors with unexpected token: EOF.

My bad, sorry.
The next line in the file was a puts, so the compiler was probably using that as an argument to “+”. Please ignore that part of my post.

1 Like

That’s expected behaviour. The &.test syntax just exands to calling test on the first argument of the block. So reduce(&.test) is equivalent to reduce { |acc| acc.test }. The second argument is never used by this block.

There is currently no way to use the short block argument syntax with more than one argument. There has been discussions about adding a short syntax for referencing positional block arguments, as introduced in Ruby 2.7 (RFC: `&.` within a block - #12 by vlazar).
That feature would allow using short argument syntax in the way you describe:

[1,2].reduce(&.+(_2))

Fair enough. I expected array.reduce(&.+) in Crystal to work the same as array.reduce(&:+) in Ruby. I guess that’s a function of Crystal being so similar to Ruby.

It would be a shame to have to resort to that ugliness for a simple case like this where the number of block arguments matches the number of arguments of the method (taking the first one as the object, that is).

[1, 2].sum

Problem solved.

Yeah, they look very much alike and have a similar purpose. But the behaviour is very different. The Crystal variant is actually more powerful which means it needs more explicitness for some use cases.

In Ruby &:foo refers to a method name. It is interpreted as a call to Symbol#to_proc. This method returns a proc which calls method foo on its first argument and passes the following arguments to that method.

In Crystal, &.foo is actually a method call - in this case without arguments. But you can also pass arguments directly to the method, for example &.foo("bar"). This is a very useful feature and not possible with Ruby’s approach. In turn, it makes it harder to pass block arguments to the method. It couldn’t work like in Ruby anyway because Crystal doesn’t have dynamic dispatch.

Thanks @asterite, I am aware of Array#sum, but mind that I’m raising a different set of concerns.
The general question, beyond #reduce, is does it make sense to allow shorthand syntax when the expected proc taks two arguments or more?
The other question is does it make sense to define #+ and #- on Number? Are there other languages taking a similar approach?

I suppose so. Block arguments don’t need to be consumed. It’s perfectly valid to ignore the following arguments, which is often used for example by iterators which pass in the index as second argument. That’s perfectly fine. Only in the case of short argument syntax it might be mildly confusing, but really only when you’re spoiled by Ruby. But when you’re aware that reduce(&.foo) is just the parameter-less version of reduce(&.foo()) it should be clear that no block arguments will be passed forward.

These methods implement the unary operators which we certainly want to exist.

But when you’re aware that reduce(&.foo) is just the parameter-less version of reduce(&.foo()) it should be clear that no block arguments will be passed forward.

Fine by me, but what is a scenario where calling reduce with a proc consuming only one arg makes sense? I cannot think of any, so it looks to me like we’re making it extremely easy to introduce bugs in one’s code. Anyhow, I don’t mean to drag this any further, if Crystal users are not bothered by this. I’m just hoping you see where I’m coming from.

These methods implement the unary operators which we certainly want to exist.

Ah, I got confused. For a moment I thought a = 3 + was a valid statement. It isn’t :relieved:
To summarise, +3 is just syntactic sugar for 3.+, which makes perfect sense, sorry for the misunderstanding.

The shorthand syntax is just syntax sugar: it’s expanded by the compiler to the longer form. When it’s time to analyze the program, the compiler has no idea whether you wrote reduce(&.+) or reduce { |arg| arg.+ } because they are exactly the same. In fact, that’s why we say it’s a shorthand syntax. If not, it would be something different.

The other question is does it make sense to define #+ and #- on Number? Are there other languages taking a similar approach?

Ruby does that, although they are called @+ and @-, but just because Ruby doesn’t have method overloading.

I also mentioned sum because the only cases of reduce(&.operator) that come to mind are + and * and those are already covered with sum and product. So the shorthand syntax is pretty much useless with reduce.

1 Like

For reduce, there’s obviously meaningful use case for a block that uses only the first arg. So the bottom line is, you effectively can’t use the short syntax for that. (unless we introduce a means to reference positional block arguments as suggested in RFC: `&.` within a block - #12 by vlazar)

Surely the issue is the semantics of “&.method” when it has no arguments, rather than “dynamic dispatch”. The compiler knows how many arguments are being passed to the block, so if the semantics were changed such that no arguments to “&.method” means “use them all”, then the above would work like Ruby. Whether this is desirable or not is another question. Then if you really want to force no arguments to the method you could always say “&.method()”, right?

lbarasti:
does it make sense to define #+ and #- on Number?

Unary #- allows you to do [1,2,3].map(&.-)

Technically, this could work. But it would be a bad idea for semantics to depend on the absence of parenthesis in a method call. That would be different from anywhere else in Crystal.

Ruby 2.7 also has a new ... syntax for arguments forwarding https://blog.saeloun.com/2019/12/04/ruby-2-7-adds-new-operator-for-arguments-forwarding.html#the--operator

Maybe something like this could be useful in blocks too.

Actually I have used reduce(&:method) in Ruby a few times, calling my own methods. Reduce is really neat for some string processing, sort of like join but where you need to do extra stuff along the way. Reduce is nice & concise - in particular it is useful to avoid the need for special handling of either the first or last element. I just can’t use the shortcut syntax in Crystal.

I guess it depends on how a coder looks at the syntax “&.method”. If you think of the “&” as a positional parameter of the block, then I agree with you because the syntax then looks like a method call, and I guess in future it could be rewritten as “(_1).method”. However if you consider “&.” to be an operator which conceptually wraps method somehow so that it is called in place of yield then it makes sense to pass all the arguments. I guess Ruby doesn’t have this issue using “:” instead of “.”, and maybe a different syntax like vlazar suggests is a good idea.