[RFC] Annotations metadata declaration/DSL

I recall that when annotations were first announced, the bodies of the annotation declaration were intentionally left blank for future expansion. Seeing as they’ve become much more useful and prevalent since then (JSON::Serializable, Athena routing, etc.), I think now would be a great time to start adding some more compiler verification metadata to them.

Judging from the discussion in this issue and from how other languages handle annotations, I think there are two main candidates for metadata:

  • Target (class, method, lib, etc.)
  • Parameters

This does raise the question of syntax. Should Crystal go down the path of Java with its meta-annotations or should it start doing compiler DSLs (á la Kotlin contracts)?

# meta-annotation
@[Target(:class, :method)]
annotation Foo
  str : String
  num : Int32 = 10
end

# DSL
annotation Foo
  target :class, :method
  
  param str : String
  param num : Int32 = 10
end

(Maybe types need to be specified as their ASTNode cousins: StringStringLiteral, Int32NumberLiteral?)

This might also open the door for processing logic to be put in the body as another macro:

annotation Bar
  target :ivar

  # possible syntax for unnamed parameters?
  params(TypeNode)

  macro annotated(target)
    {% if target.type != nil %}
      def {{ target.name }}: {{ params[0] }}
        # ...
      end
    {% end %}
  end
end
1 Like

Personally I love the DSL route, I also like the idea of handling all annotation logic inside of the annotation itself with something like that annotated macro. It just makes more sense than having to write a separate macro outside of the annotation that looks for annotations.

I kind of like how PHP handles it

/**
 * @Annotation
 * @Target({"PROPERTY","METHOD"})
 */
final class SomeAnn
{
    /**
     * @var string
     */
    public $name;

    public function __construct(array $values)
    {
        if (!isset($values['value']) || !\is_string($values['value'])) {
            throw new RuntimeException(sprintf('"value" must be a string.'));
        }

        $this->name = $values['value'];
    }
}

Or the crystal version would be like

annotation SomeAnn

  getter name : String

  # Initialize is optional, gets passed a Tuple and NamedTuple supplied
  def initialize(positional_args, named_args)
    # Can do some custom stuff here.  Otherwise properties must be defined as named arguments.
  ned
end

One good way I came up with to handle annotations currently is to define a class/struct that represents the annotation. Then you can iterate over the annotations doing like MyAnnClass.new {{ann.named_args.double_splat}}; which handles required properties, and default values etc.

Excessive annotation syntax infiltrating the language will slowly diminish Crystal’s beautiful syntax.

2 Likes

I think we could declare it like annotation Foo(args) because when you use it it’s exactly like a call, minus the block. For example:

module JSON
  annotation Field(*, key = nil, ignore = false, root = nil, converter = nil, presence = false, emit_null = false)
end

Type restrictions should be HashLiteral, StringLiteral, etc., because annotations operate on AST nodes. However, I’m not sure how useful this is because it’s hard to specify the types of keys and values. Or for example converter can be a type, a method, etc.

Then I don’t know about the annotated hook, we’d have to think of real use cases.

At the very least, I think it would be really helpful for documentation. I recently implemented DB::Serializable for crystal-db and I noticed that the only way to really describe the possible parameters for the annotation is through manually listing them inside a doc comment. Personally, I think it would’ve been much cleaner to do something like

module DB
  annotation Field(ignore : BoolLiteral = false, key : StringLiteral | NilLiteral = nil, converter : TypeNode | NilLiteral = nil)
end

and have crystal doc create some kind of documentation for it. At that point, I see no reason not to typecheck.

2 Likes

Bumping this. For what I do this is something I struggle with quite often. It’s quite annoying when you want to use an annotation for something, but then need to also have another type to represent the data that should be read off the annotation. Making the type that gets applied AND the type that stores/handles the data, the same type would be :100:.

IMO this would be the most common use case, I still think ideally annotations should work more like:

@[Annotation]
struct MyAnn
end

as opposed to:

annotation MyAnn
end

Essentially this would allow that type (either a class or struct) to be applied via the @[...] syntax. In macro land the type of MyAnn would be Annotation like it is now to allow reading the arguments off the annotation. But, a new method could also be added, like build or something, that would create an instance of the type based on arguments defined as part of the annotation. For example:

@[Annotation]
record MyAnn, name : String, active : Bool = true

@[MyAnn(name: "foo")
def foo
  {{ @def.annotation(MyAnn)[:name] }}
end

foo # => "foo"

@[MyAnn(active: false)
def bar 
  "bar"
end

bar # => Compile error because `name` wasn't supplied and it's not nilable nor has a default value

@[MyAnn(name: "baz")]
def baz
  {{ @def.annotation(MyAnn).build }}
end

baz # => MyAnn(@name="baz", @active=false)

IMO this would be a game changer as it would make annotations less unique, and more like standard types, supporting modules/inheritance etc.

I made an issue about this: https://github.com/crystal-lang/crystal/issues/9802.

A use case for macro annotated, precisely, is eliminating the need for additional DSL powered by the language itself. For example, we could simply write:

annotation Flags
  macro annotated(target, *args, **opts)
    {% unless target.is_a?(EnumDef) %}
      {% raise "can only apply @[Flags] to enums" %}
    {% end %}
  end
end

Then we could allow annotations to include or use an implementation for target, taking advantage of the fact that multiple hooks are all run and never override each other (so that we could still have our own annotated hook):

module AnnotationHelper
  macro target(*kinds)
    macro annotated(target, *args, **opts)
      \{% kinds = {{ kinds }} %}
      \{% if target.is_a?(ClassDef) && !kinds.includes?(:class) %}
        \{% raise "expected one of: #{kinds.map(&.id).join(", ").id}, got ClassDef but no :class" %}
      \{% elsif target.is_a?(EnumDef) && !kinds.includes?(:enum) %}
        \{% raise "expected one of: #{kinds.map(&.id).join(", ").id}, got EnumDef but no :enum" %}
      \{% elsif target.is_a?(Def) && !kinds.includes?(:method) %}
        \{% raise "expected one of: #{kinds.map(&.id).join(", ").id}, got Def but no :method" %}
      \{% end %}
    end
  end
end

annotation Flags
  include AnnotationHelper
  target :enum

  # less terse, compiler still needs to permit macro expansions in annotation scope
  AnnotationHelper.target :enum
end

The contents of the annotated hook should be interpolated after target in the same scope. One of my use cases is to automatically generate overloads to circumvent Crystal’s current overload ordering behaviour:

# :nodoc:
annotation ExpandPrecRnd
end

module Math
  @[ExpandPrecRnd]
  def sqrt(value : BigFloatR, *, precision : Int = BigFloatR.default_precision, mode : BigFloatR::RoundingMode = BigFloatR::RoundingMode.default)
    BigFloatR.new(precision: precision) { |mpfr| LibMPFR.sqrt(mpfr, value, mode) }
  end

  {% for a_def in @type.methods %}
    {% if a_def.annotation(ExpandPrecRnd) %}
      def {{ a_def.name }}({{ a_def.args[0...a_def.splat_index].splat }})
        {{ a_def.name }}(
          {{ a_def.args[0...a_def.splat_index].map(&.name).splat }},
          precision: BigFloatR.default_precision,
          mode: BigFloatR::RoundingMode.default,
        )
      end
    {% end %}
  {% end %}

  # the above generates:
  # ```
  # def sqrt(value : BigFloatR)
  #   sqrt(value,
  #     precision: BigFloatR.default_precision,
  #     mode: BigFloatR::RoundingMode.default,
  #   )
  # end
  # ```
  # because the original def is _less_ specific than the `def sqrt(value)`
  # overload in stdlib (the one without parameter restrictions)
end

This approach is rather imperative, as that inline macro must be copied in every scope that uses ExpandPrecRand. An annotated hook would make this declarative:

annotation ExpandPrecRnd
  macro annotated(target, *args, **opts)
    # extra error handling (e.g. `args.empty?`, `target.is_a?(Def)`)

    def {{ target.name }}({{ target.args[0...target.splat_index].splat }})
      {{ target.name }}(
        {{ target.args[0...target.splat_index].map(&.name).splat }},
        precision: BigFloatR.default_precision,
        mode: BigFloatR::RoundingMode.default,
      )
    end
  end
end

module Math
  @[ExpandPrecRnd]
  def sqrt(value : BigFloatR, *, precision : Int = BigFloatR.default_precision, mode : BigFloatR::RoundingMode = BigFloatR::RoundingMode.default)
    BigFloatR.new(precision: precision) { |mpfr| LibMPFR.sqrt(mpfr, value, mode) }
  end

  # automatically generates the same def as the previous snippet
end

I am not a fan of declaring the annotation parameters right next to the annotation name, since to me that implies each annotation may only have one “overload”. Likewise, I don’t think we should impose a one-to-one correspondence from annotations to “real” Crystal types; this prevents users from mapping different annotations to the same type for e.g. performance reasons, but also this correspondence can probably be implemented in 100% user code if we allow user macros in annotations instead.

Slightly related to the last point is that you can already define methods on annotation metaclasses:

annotation A
end

def A.foo(x, y)
  x + y
end

A.foo(1, 2) # => 3
5 Likes