[RFC] Standardized Abstractions

Standardized Abstractions

Extracting URI::Params is a struct? Why? - #10 by Blacksmoke16 to its own topic.

Background

Using decorators is usually a better approach when you want to customize/extend a type, especially for ones you do not own. For example:

struct CustomRequest
  getter request : HTTP::Request

  forward_missing_to @request

  def self.new(method : String, path : String, headers : HTTP::Headers? = nil, body : String | Bytes | IO | Nil = nil, version : String = "HTTP/1.1") : self
    new HTTP::Request.new method, path, headers, body, version
  end

  def initialize(@request : HTTP::Request); end

  def safe? : Bool
    @request.method.in? "GET", "HEAD", "OPTIONS", "TRACE"
  end
end

This way you can do something before/after a specific method, or add more methods, all the while not touching anything related to the upstream type itself.

HOWEVER, with the way the stdlib is implemented at the moment, this doesn’t work as well as it could since everything is typed as concrete implementations instead of abstractions. This means you cannot have this custom request type work as a drop in replacement to HTTP::Request , even if it is 100% compatible with it.

PHP’s PSR

PHP handles this via PHP Standards Recommendations - PHP-FIG, which define interfaces/requirements for various common libraries/implementations, such as eventing, caching, and HTTP clients, factories, and messages. This allows framework/application developers to design things such that any implementation could be used. E.g. an HTTP Client from Symfony, Guzzle, or even a custom implementation.

Ultimately this makes the user happier given they can use what they want and makes the developer’s life easier as they do not need to make a bunch of adapters or try and make things work with an arbitrary amount of implementations.

Composer’s provide

PHP’s package manager (composer) also taps into this feature via its provide key, which essentially allows defining a dependency like psr/http-client-implementation that would be satisfied if a library that is installed “provides” that package. E.g. one from the list of Packagist.

Proposal

I propose to make the stdlib less coupled with its concrete implementations by doing something similar to PHP’s PSR by introducing some interfaces (modules) that describe the public API that some common types must implement. This will allow using decorators as a drop in replacement, assuming they correctly implement the interface. E.g. include RequestInterface.

There are a few choices in regards to the exact implementation, which can be hashed out later.

Benefits

  • Stdlib better follows the SOLID principles
  • Provides a standard for shard creators and consumers
    • I.e. imagine using crest or halite interchangeably with HTTP::Client. Such as within OAuth2::AccessToken#authenticate among other places
  • Provides a place to define/document requirements/suggestions for implementers, such as that HTTP headers MUST be case-insensitive

Challenges

  • Overhead of determining the API and updating all usages in stdlib
  • How to distribute/provide the interfaces
  • Possibly somewhat breaking given not every method on HTTP::Request would be on the interface most likely
  • Not everyone may be familiar with SOLID, interfaces, or benefits thereof
5 Likes

I fully agree both on the proposal, it’s benefits and on the challenges it brings.

My opinion is that the benefits of such a good move easily overcome the challenges.

What is important is to build an easy path to help those who may be not familiar with these concepts, like SOLID, interfaces and etc.

In other words, the secret sauce is how we make this, with focus on make it understandable, with good documentation and, most important, for humans.

2 Likes

I really like Dart’s implicit interfaces for this. Downsides compared to your proposal of course are adding language and compiler complexity as well as feasibility of something like that with an open type system.

1 Like

Oh, that’s an interesting concept. So you can extract the interface from any type on the fly. And implementing types are accepted in the same way as the original type.
This could be useful for its flexibility. It works without extracting an abstract interface, which would require cooperation from the owner of that type as well as consumers.

Also Structural subtyping can be considered, where names doesn’t matter. A concrete type is a member of a structural category if it happens to have all the needed elements, no matter what it happens to be called.

Structural typing is very similar to duck typing from dynamic programming languages, except that type compatibility is determined at compile time rather than at run time. If you’ve used Go, you’ll recognize that Go interfaces are structural types.

Major benefit is You do not declare structural relationships ahead of time, instead it is done by checking if a given concrete type can fulfill the required interface.

Just my two cents :smile:

4 Likes

This is pretty neat. But yea, is a lot to think about, like if someone monkey patches something in, what happens? I’m also not a huge fan that it would add everything to the interface. I.e. normally the type adds more helper methods and such than the base interface. This could also be an issue if/when more methods are added to the type, say HTTP::Client for example, as that would mean every implementation of it would need to add them?

This can work in some cases, but falls down when you want to have an ivar/class var store an instance of some interface, which you need the interface for to type it.

Ultimately I still think a decoupled dedicated interface is the most robust way forward. Just because it would allow refactoring or renaming of the core stdlib types without affecting the other users/implementers. If it was released as its own package(s), would also allow versioning the interface separately from the core types, which could be useful as, like everything else, these interfaces do not need to be static and can evolve over time.

They only inherit public API of course. Then I think typical usecases for this would be a kind of decorator where you would wrap an instance of the original type anyhow, so implementing all those methods could usually mean just forwarding them all to the original. Using macros or compiler aware forward_missing_to that then could be fairly low on boilerplate.

Yea I think that would be perfect for a decorator, as to your point, you’d essentially want a clone of that type’s API. But I’m still hesitant that it would be the best option for general abstractions, as the core idea of providing them is that you could use any implementation, not just those based on the core stdlib type, which would include additional helper methods which probably don’t need to be on the core interface.

Maybe we could do both? Something something helper macro/type/syntax for decorating a type via its implicit interface, while still providing explicit ones to handle the other use cases? Granted both of them do not have to be implemented at the same time.

Judging from the OP, none of this requires a change to the core language, right? How is this different from simply adding more modules / abstract classes?

1 Like

Exactly. The simplest solution to this is to come up with some modules, either in the stdlib or their as their own shard, that defines the API of some common types. Then include that module into the stdlib implementation, finally updating usages of the concrete stdlib type with that module (interface).

Anyone using say HTTP::Client would be backwards compatible as the type isn’t changing. But could/should update certain cases to use the interface, and library authors could update their libs to include it as well.

What needs to be hashed out still is like:

  1. What types do we want to do this with, and how far/specific we want to get
  2. What the API for each type should be
  3. How to distribute/provide the interfaces
  4. How to document/explain the reasoning for doing it

I’ll look more into some of the PSRs and come up with the Crystal equivalents, then can go from there.

2 Likes

Okay, so a fairly direct transaction of PSR-7 to Crystal would look something like: [RFC] Standardized Abstractions · GitHub.

The main change is that all their types are immutable, but not really sure I see the benefit there, so I just made everything be your standard getter/setters. They also have file uploads built directly into things, hence the UploadedFileInterface. Might not hurt to have a standardized way to represent uploaded files, even if the stdlib does not provide that ability by default.

There are also a few methods that are either new, or might be a bit tricker in Crystal, mainly on the ServerRequestInterface, specifically the #parsed_body and #attributes methods. Probably don’t need to ultimately have those on ours.

They also have a URIInterface which basically represents the stdlib’s URI type. Can’t really think of a use case where you’d need/want to define your own, so could just skip that and use URI directly, or even go a bit lower and have getter/setters for each part instead of requiring a URI instance.

1 Like