Please explain this error

I was experimenting with some code.
These compile.

cnt = seg[..(kn - 1) >> s].map { |m| (~m).popcount }.sum

cnt = seg[0..(kn - 1) >> s].map { |m| (~m).popcount }.sum

cnt = seg[0i32..(kn - 1).to_i32 >> s].map { |m| (~m).popcount }.sum

But these don’t

cnt = seg.to_unsafe[0..(kn - 1) >> s].map { |m| (~m).popcount }.sum

cnt = seg.to_unsafe[0i32..(kn - 1).to_i32 >> s].map { |m| (~m).popcount }.sum

And gives this. error for both.

➜  crystal build twinprimes_ssozgisty.cr -Dpreview_mt --release
Showing last frame. Use --error-trace for full trace.

In /home/jzakiya/crystal/share/crystal/src/pointer.cr:117:11

 117 | (self + offset).value
             ^
Error: no overload matches 'Pointer(UInt64)#+' with type Range(Int32, Int32)

Overloads are:
 - Pointer(T)#+(offset : Int64)
 - Pointer(T)#+(other : Int)

It is hard to fully know as you don’t provide enough information (what is seg? what do you get if you add --error-trace when you compile?), but seg.to_unsafe likely returns a pointer, which define [] as

  def [](offset)
    (self + offset).value
  end

which would then trigger your error.

What do you expect to happen if you add a range to a pointer? I’d expect the compilation error you are getting.

Well, a ranged pointer, i.e. an Array? I think the semantics makes sense, less so the operator overload.

seg = Slice(UInt64).new(((ks - 1) >> s) + 1)

It seems odd the same line with bounds checking will compile but won’t compile when bounds checking turned off: seg.to_unsafe[range]

Without knowing the motivations behind it, I would consider this a bug.

What do you expect the return type of asking a subrange of a pointer? That doesn’t make a lot of sense. With a pointer you can only point to a single place, not a range, exactly because the pointer has no bounds.

In general I would avoid not doing bound checks. In all my tests avoiding this leads to the same performance, not a better performance. I don’t know why, maybe it’s branch prediction.

As you can see, I’m reading over a range in all cases.
It compiles and runs fine as I initially show, but using .to_unsafe causes an error.
As a user, this is surprising, because I would expect if it compiles with bounds checks it should compile with no bounds checks (I thought that’s what .to_unsafe does). I had|have no knowledge of the implementation details (pointers, etc), but I didn’t expect this behavior, as it worked everywhere else I used it.

I have code sections, particularly in inner loops, where doing read|writes using .to_unsafe produces substantial performance benefits. I normally don’t use it willy-nilly, but it DOES make a difference in certain places. (I first observed this in the Rust version of the same code when turning off bounds checking in it.)

To clarify, to_unsafe isn’t equivalent to “I want to do things without bound checks”

Maybe that’s why your expectations don’t match what ends up happening.

OK, but I’m just trying to understand why the later code doesn’t compile.
Doesn’t to_unsafe not do bounds checking?
Isn’t that why it’s faster for the uses I cited?
Or where, or for what other purposes, would you use it?
Again, I’m just trying to understand what’s going on.

def **to_unsafe** : [Pointer](https://crystal-lang.org/api/1.3.2/Pointer.html)(T) [#](https://crystal-lang.org/api/1.3.2/Array.html#to_unsafe%3APointer%28T%29-instance-method)

Returns a pointer to the internal buffer where `self` 's elements are stored.

This method is **unsafe** because it returns a pointer, and the pointed might eventually not be that of `self` if the array grows and its internal buffer is reallocated.

ary = [1, 2, 3]
ary.to_unsafe[0] # => 1

Right, to_unsafe returns a Pointer, as you mention. If you check the available methods of Pointer there’s no #[](Range) overload, simply because such overload doesn’t make sense.

Doesn’t to_unsafe not do bounds checking?

No, to_unsafe returns a Pointer.

Isn’t that why it’s faster for the uses I cited?

I don’t have a proof that it’s faster.

Or where, or for what other purposes, would you use it?

I would never use it unless I need to interface with C.

1 Like

Some methods in Pointer still have range semantics, e.g. Pointer#map! implements Slice#map! and not the other way round, which may be where the confusion comes from. I think we should drop those instance methods from Pointer.

3 Likes