Method chaining metaprogramming

Help & Support
1

I recognize this is pretty long. I actually spent like 3 hours trying to make it more concise and I can’t do that and still communicate what I’d like to do and what’s keeping me from doing it.

Generic type inference (class Thing(A, B, C) where A, B, and C are inferred from values passed to the constructor on instantiation) is currently an all-or-nothing deal. So you either specify Thing(One, Two, Three).new(a, b, c) or Thing.new(a, b, c) — there is no “partial” generic type inference (abbreviating as GTI from now on). So I’m trying to figure out if there’s a way to do achieve something with the current implementation (and if not, could/should partial GTI support be added to Crystal?).

I’m writing an ORM called Interro and loading other models via JOINs has become a bit of a struggle to do in a way that still allows chaining easily.

What I’d like to be able to do is this:

orders = OrderQuery.new
  .with_customer
  .with_delivery
  .with_vendor

# Block yields Tuple(Order, Customer, Delivery, Vendor)
orders.each do |(order, customer, delivery, vendor)|
  # ...
end

… where each those methods are all defined on OrderQuery. Adding with_customer will add another object to what the block yields, so removing .with_vendor will yield {Order, Customer, Delivery}.

I’m currently attempting to implement the chaining via a Projection type, instantiated via a method on OrderQuery's superclass:

struct Projection(ModelTypes, Delegate, OtherQuery, OtherModelType)
end
Current implementation (does not work) 
protected def project(other : QueryBuilder(U), as name : String) forall U
  {% if T < Tuple %}
    Projection({*T, U}, self, QueryBuilder(U), U).new(self, other, name)
  {% else %}
    Projection({T, U}, self, QueryBuilder(U), U).new(self, other, name)
  {% end %}
end

struct Projection(ModelTypes, Delegate, Other, OtherModelType) < QueryBuilder(ModelTypes)
  def initialize(@delegate : Delegate, @other : Other, @relation_name : String)
  end

  def sql_table_name
    @delegate.sql_table_name
  end

  def sql_table_alias
    @relation_name
  end

  def set_delegate(@delegate)
  end

  macro method_missing(call)
    # We can mutate the delegate. Since it's a struct, the instance that was
    # passed in was copied, so it's not shared with any other object. We own
    # this instance.
    @delegate.distinct = @distinct
    @delegate.join_clause = @join_clause
    @delegate.where_clause = @where_clause
    @delegate.order_by_clause = @order_by_clause
    @delegate.offset_clause = @offset_clause
    @delegate.limit_clause = @limit_clause
    @delegate.args = @args
    @delegate.transaction = @transaction
    %new_query = @delegate.{{call}}

    %new_projection = Projection({*ModelTypes, OtherModelType}, self, Other, OtherModelType).new(
      delegate: self,
      other: @other,
      relation_name: @relation_name,
    )
    %new_projection.distinct = %new_query.@distinct
    %new_projection.join_clause = %new_query.@join_clause
    %new_projection.where_clause = %new_query.@where_clause
    %new_projection.order_by_clause = %new_query.@order_by_clause
    %new_projection.offset_clause = %new_query.@offset_clause
    %new_projection.limit_clause = %new_query.@limit_clause
    %new_projection.args = %new_query.@args
    %new_projection.transaction = %new_query.@transaction

    %new_projection
  end

  def select_columns(io : IO) : Nil
    @delegate.select_columns io
    io << ", "
    @other.select_columns io
  end
end
  • ModelTypes is the type of the Tuple yielded to the block (see code example near the end)
  • Delegate is the QueryBuilder type that the projection delegates to with method_missing. It’s the same pattern I’ve been using in the “background details” below.
  • OtherQuery is how we get the other table name and things to query against. If you think of the join as table_a JOIN table_b, Delegate provides table_a and OtherQuery provides table_b.
  • OtherModelType is the model type that OtherQuery wraps. It gets added to the derived Projection with {*ModelTypes, OtherModelType}.

ModelTypes is the only type whose instances aren’t being passed to the constructor — we don’t get instances until we get query results back). Delegate, OtherQuery, and OtherModelType are so they could be inferred if Crystal supported partial GTI.

Another complication is that, in most cases, methods on OrderQuery (the Delegate in the example above) are just scopes and will return another OrderQuery instance. But the with_* methods end up returning other Projection instances, so we can’t pass OrderQuery as the Delegate type, but I’m not sure what other options I’ve got here.

This is where I was going with the pGTI idea. If we could infer everything but ModelTypes, the return value of @delegate.{{call}} in method_missing could be inferred. But I’m not 100% sure even that would work because AFAIK the functionality doesn’t exist for me to test it. :joy:

Some more background details so you can see where I'm coming from

My initial implementation allowed for things like this:

OrderQuery implementation 
struct OrderQuery < Interro::QueryBuilder(Order)
  # Get the customer for each order
  def with_customer
    # Get the Order attributes and the Customer attributes into the `SELECT` clause
    columns = String.build do |str|
      select_columns str
      str << ", "
      CustomerQuery.new.select_columns str
    end

    self
      .inner_join("customers", on: "orders.customer_id = customers.id")
      .fetch(column_names, as: {Order, Customer})
  end
end

The result is that your application code can do things like this:

orders = OrderQuery.new.with_customer

# Note the customer is yielded to the block
orders.each do |(order, customer)|
  # ...
end

If you chain more methods on the result of the fetch call, they’re delegated to the OrderQuery so you can still sort the results or scope the result set down further. But things get weird if you want to do another fetch to tack on more models to each result row.

One thing I’ve been doing in the interim is just creating separate QueryBuilder types for each set of JOINs I want to do. The fetch / DynamicQuery implementation was an attempt to streamline some of that, but now I’m trying to get it to expand beyond a single JOIN.

1 Like
2

Does something like this help?

class Order
  def initialize(@id : Int32)
  end

  def fetch(t : Customer.class)
    Customer.new
  end

  def fetch(t : Delivery.class)
    Delivery.new
  end
end

class Customer
end

class Delivery
end

struct OrderQuery(T)
  def self.new
    OrderQuery(typeof(Tuple.new)).new([Order.new(1), Order.new(2)])
  end

  def initialize(@orders : Array(Order))
  end

  def each
    {% begin %}
      @orders.each do |order|
        yield(
          order,
          {% for t in T %}
            order.fetch({{t}}),
          {% end %}
        )
      end
    {% end %}
  end

  def with_customer
    _with(Customer)
  end

  def with_delivery
    _with(Delivery)
  end

  def _with(t : U.class) forall U
    {% begin %}
      OrderQuery(
        {
          {% for t in T %}
            {{t}},
          {% end %}
          U,
        },
      ).new(
        @orders
      )
    {% end %}
  end
end

puts "--- Order ---"
query = OrderQuery.new
query.each do |order|
  p order
end

puts "--- Order + Customer ---"
query = OrderQuery.new.with_customer
query.each do |order, customer|
  p order, customer
end

puts "--- Order + Customer + Delivery ---"
query = OrderQuery.new.with_customer.with_delivery
query.each do |order, customer, delivery|
  p order, customer, delivery
end

puts "--- Order + Delivery + Customer ---"
query = OrderQuery.new.with_delivery.with_customer
query.each do |order, delivery, customer|
  p order, delivery, customer
end
3 Likes
3

It looks like it might! I’ll try to work this approach in today and see how it goes. Thanks, @asterite!

1 Like