Updates in the Athenaverse - New Component | Proxy Support | Form Data Deserialization

News
1

Updates in the Athenaverse

It’s been a while since the last update, but the wait is over! This release cycle in the Athenaverse focuses on continuing to evolve the framework:

  • Initial release of the new MIME component
  • Native support for Proxy & Load Balancers
  • Native support for Trusted Hosts
  • Normalized exception types
  • Refactored query param implementation along with native support for deserializing application/x-www-form-urlencoded bodies via ATHR::RequestBody

The MIME Component

The highlight of this cycle is the introduction of a new MIME component that provides an RFC compliant way to create and manipulate MIME messages, or as they’re better known as, emails:

email = AMIME::Email
  .new
  .from("me@example.com")
  .to("you@example.com")
  .cc("them@example.com")
  .bcc("other@example.com")
  .reply_to("me@example.com")
  .priority(:high)
  .subject("Important Notification")
  .text("Lorem ipsum...")
  .html("<h1>Lorem ipsum</h1> <p>...</p>")
  .attach_from_path("/path/to/file.pdf", "my-attachment.pdf")
  .embed_from_path("/path/to/logo.png")

This component does not handle sending emails, but is intended to act as a base for future Athena components that could do so. It of course may also be used by other shards who would like a stable/standardized way of representing emails to be sent such that they could be inner-operable with any other shard also using it.

Proxy & Load Balancer

It is a common practice to host a web application behind a load balancer or reverse proxy. This by itself is not a problem for Athena. However it does allow a way for that extra metadata to be exposed, which is not currently exposed on the request object. See Expose connection metadata in HTTP::Server::Context · Issue #5784 · crystal-lang/crystal · GitHub, Why HTTP::Request uri is private and scheme · Issue #7096 · crystal-lang/crystal · GitHub, and [RFC] Effective request URL for `HTTP::Server` · Issue #10246 · crystal-lang/crystal · GitHub for more details.

Even if it was, they would not expose the correct values since they’d be based on the reverse proxy/load balance and not the client’s original request. This release brings support for the various x-forwarded-* Proxy Headers.

For example to to allow reading the port and scheme from requests originating from proxies under your control:

ATH.configure({
  framework: {
    # The IP address (or range) of your proxy.
    trusted_proxies: ["192.0.0.1", "10.0.0.0/8"],

    # Trust only `x-forwarded-port` and `x-forwarded-proto` headers.
    trusted_headers: ATH::Request::ProxyHeader[:forwarded_port, :forwarded_proto]
  },
})

The implementation also supports dynamic IPs and custom header names for proxies that do not use the x-forwarded-* headers. See Proxies & Load Balancers - Athena for more details.

Trusted Hosts

In a similar vein, this release also allows defining the host names that are considered trusted. This effectively prevents host header attacks. When set, requests whose hostname does NOT match any of the provided patterns will recieve a 400 response.

Normalized Exception Types

Many of the Athena components defined an Exceptions namespace that custom exception types were included in. Some components additionally defined a base exception type like ConsoleException to use as the base of all other. The idea being the user would be able to rescue it in order to handle any exception raised from within the component itself. The problem there was that it prevented the custom error types from inheriting related stdlib exception types, such as ArgumentError.

With the release of Allow rescuing exceptions that include a module by Blacksmoke16 · Pull Request #14553 · crystal-lang/crystal · GitHub, all of the components had their exception namespace normalized to be Exception, and this module is then included in all custom exception types. Meaning it is now possible to do things like:

  1. Rescue ACON::Exception to handle any exception type defined by the component itself
  2. Rescue ArgumentError to handle any exception resulting from unexpected/invalid data either from the component, or Crystal stdlib
  3. Rescue ACON::Exception::InvalidArgument to rescue only unexpected/invalid data exceptions from the component only

Ultimately this makes error handling a bit more standardized and flexible depending on the exact need.

Refactored query param / form data deseralization support

Last but not least this PR completely refactors how query parameters and form data is handled. Previously the implementation was its own disparate thing. It is now implemented within the controller argument value resolver implementation. This ultimately makes things easier to maintain.

Query parameters are now handled by annotating the related parameter directly, instead of the controller action method.

class ExampleController < ATH::Controller
  @[ARTA::Get("/")]
  def index(@[ATHA::MapQueryParameter] page : Int32) : Int32
    page
  end
end

Additionally with the introduction of Add `URI::Params::Serializable` by Blacksmoke16 · Pull Request #14684 · crystal-lang/crystal · GitHub into the stdlib, the ATHR::RequestBody value resolver now natively handles x-www-form-urlencoded request payloads:

record LoginDTO, username : String, password : String do
  include URI::Params::Serializable
end

class ExampleController < ATH::Controller
  @[ARTA::Post("/login")]
  def login(@[ATHA::MapRequestBody] login : LoginDTO) : Nil
    # ...
  end
end

Which can in turn have validation constraints attached to them.

Lastly, the new ATHA::MapQueryString annotation can do a similar thing, but for the query string of a request:

record PaginationContext, page : Int32 = 1, per_page : Int32 = 100 do
  include URI::Params::Serializable
end

class ArticleController < ATH::Controller
  @[ARTA::Get("/articles")]
  def articles(
    @[ATHA::MapQueryString]
    pagination_context : PaginationContext,
  ) : Array(Article)
    # ...
  end
end

Such that you have a more reusable, testable, and flexible setup than using many ATHA::MapQueryParameter annotations.

Bonus Section! Improving DX

Code Coverage

As primarily a one person developer, I rely heavily on Athena’s test suite to ensure things are working as expected as I make changes. A behind-the-scenes track of work this cycle was a push to better integrate code coverage reporting is a direct result of this. The idea being it provides numbers to back of the fact that Athena is well tested.

The result of all this work is I’m pleased to say Athena’s test coverage is ~97.5% at time of writing. This can be explored in more detail on Codecov. PRs are welcomed to improve this further ;) CI is setup to soft-require 100% coverage on new code, but of course there are some cases where this is not possible, so mostly serves as a guideline not a hard requirement.

There are also some outstanding issues like Missing DWARF debug information · Issue #15360 · crystal-lang/crystal · GitHub that would likely improve coverage further as Crystal’s debug information gets more robust.

However, while this is good and all, the next major hurdle would be integrating Macro Code Coverage · Issue #14880 · crystal-lang/crystal · GitHub when/if a solution is merged/released. This would ensure that Athena’s macro code, especially within the DI component, is working as expected as well.

Local Development

I also spent some time replacing Athena’s monorepo’s Makefile with a justfile and revamping its CONTRIBUTING.md file. So there has never been a better time to contribute!

And that’s it for this release! As usual feel free to join me in the Athena Discord server if you have any suggestions, questions, or ideas. I’m also available via Email.

3 Likes