# Abstracting out adapters - Dependency Injection

**URL:** <https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762>\
**Category:** Help & Support\
**Created:** [July 6, 2022, 5:19pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762 "2022-07-06T17:19:21Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![rob](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rob/32/1409_2.png) [@rob](https://forum.crystal-lang.org/u/rob)\
**Post date:** [July 6, 2022, 5:19pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/1 "2022-07-06T17:19:21Z")

</div>

One of the things I appreciate about the design of Shards is the ability for multiple owners to contribute their own shard for a purpose – I can build a redis adapter and share it, and so can you. And someone else. We can work together and avoid a landrush.

But how do I write a library that is capable of handling “whatever redis” you give me?

It’s easy for me to write this because I know the type of `redis_url` – a string:

```crystal
require "redis"

module Library
  class Config
    property redis_url : String
  end
end

Library::Config.new.redis_url = "redis://localhost:6379"

```

It’s also easy for me to declare that I might want `property redis_instance : Redis::PooledClient` if I know that you’re using a Redis library that provides that feature.

* * *

What I can’t find a solution for is as follows. I’d like for the user of my library to be able to pass in an instance of an unknown class, which obeys an interface:

```crystal
module Library
  class Config
    property redis_instance : ???
  end
end

Library::Config.new.redis_instance = Redis::Connection.new "redis://localhost:6379"

```

In this case, I only need the redis instances to have these methods: #hget, #hgetall, #hset, #expire, #del, #ttl, #increment, #hincrby, #flushall, #lpush, #lrem, #zcount.

- Typing `redis_instance` as a generic Class doesn’t work because the compiler doesn’t believe that the methods I need will be defined.
- Inheritance doesn’t work – this would create an N-interfaces for N-projects problem
- Duck Typing _would_ work but I don’t know how to lead the compiler, if it’s even possible.

* * *

Library flexibility isn’t the only reason I’d like to achieve this. I’d also like to share the connection pool of the host application. For other databases, this could be an important and even expensive distinction – if embedding my library in an app requires 2 connection pools, that may require someone to pay for a higher tier of a database or be faced with limited horizontal scaling because the number of connections is too high.

---

<div class="post-metadata">

**Author:** ![Blacksmoke16](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/blacksmoke16/32/1241_2.png) [@Blacksmoke16](https://forum.crystal-lang.org/u/Blacksmoke16)\
**Post date:** [July 6, 2022, 6:23pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/2 "2022-07-06T18:23:02Z")

</div>

So basically the solution to this is [Dependency inversion principle - Wikipedia](https://en.wikipedia.org/wiki/Dependency_inversion_principle). This use case is the core reason Athena provides a bunch of interface modules. It allows Athena to base its implementation on the interface itself, and not a specific concrete implementation. I.e. If you wanted to customize some specific part of it, you can define a custom implementation, include the module, and tell it to use your version and everything continues to work. Assuming you implemented the interface correctly.

Another good example are the DB drivers in core Crystal org. I.e. there is `crystal-db` that defines the “interfaces” for the various common parts of DB system. Like the connection, the driver, the query methods etc. Then each specific DB shard, requires this lib as a dependency, extending/including its interfaces to provide driver specific implementations.

Because each specific DB uses the same interfaces, they are all interchangeable (at least in regards to the abstractions setup within `crystal-db`). Some DBs have unique features that would be ofc limited to that specific implementation.

For the most part in Crystal, there is no dedicated “interface” type. The best option that works essentially just as good, is a module with abstract defs. E.g.

```auto
module RedisInterface
  abstract def hget
  abstract def hset
  
  # ...
end

```

Another option would be to go a step further, and maybe _don’t_ have an interface for redis itself, but for a higher level feature like `CacheInterface` that would allow you to use redis, or memcached, or even a DB or the file system. I’d also checkout [[RFC] Standardized Abstractions](https://forum.crystal-lang.org/t/rfc-standardized-abstractions/4414/1), as that is focused on the same sort of problem, and shows some examples of how PHP handles it. Doing something similar for Crystal may also be beneficial.

EDIT: I missed the core question. The solution to “handle any redis” is to ultimately have just a single robust redis shard everyone uses, versus some people using shard A and other B, and even others C. A workaround for this would be to have your lib depend on `RedisInterface` that it defines, then create adapters for each redis shard that implement that interface. Then the user would have to require whatever adapter they’re using (or maybe you could detect it somehow via a macro and require it for them).

@crimson-knight Went through a similar exercise, so he can prob provide some more specific insight as well.

---

<div class="post-metadata">

**Author:** ![crimson-knight](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/crimson-knight/32/1643_2.png) [@crimson-knight](https://forum.crystal-lang.org/u/crimson-knight)\
**Post date:** [July 6, 2022, 6:43pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/3 "2022-07-06T18:43:53Z")

</div>

I was just posting a lengthy reply about what I wish we could do… But yes, the exercise just showed me I still have much to learn sensei 😅

Haha just a little humor, you’ve got the patience of a saint my friend!

---

<div class="post-metadata">

**Author:** ![crimson-knight](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/crimson-knight/32/1643_2.png) [@crimson-knight](https://forum.crystal-lang.org/u/crimson-knight)\
**Post date:** [July 6, 2022, 7:52pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/4 "2022-07-06T19:52:33Z")

</div>

So, you’re right on the money that the community would need a single “redis” shard that did everything we need. But, that would mean we need a tool like Ruby Gems but for Crystal (which I’d love, but that’s a bigger hurdle to overcome I think).

I think it would be worth adding decoupling behavior to the Crystal language though. Right now it’s basically impossible to write a shard that accepts any library that would meet any interoperability requirements.

You’ve probably felt this pain at some point though, being a framework maintainer in Crystal and being required to know exactly what all your dependencies are means you have a very tightly coupled system. It would make a much more loosely coupled shard environment if we could create some kind of failsafe syntax. I have this wild idea that we could have a class definition with the required methods and some minimal default behavior in the event the required class isn’t passed in.

```crystal
module Library
  class MyClass
    property redis_client : Redis

    def initialize(@redis_client)
    end

   failsafe class Redis
     # These methods are checked against any @redis_client to ensure they respond_to? the necessary methods
     required_methods: :hget, :hset

    failsafe def initialize
      # Initialize the class in the event nothing has been passed in for the @redis_client
    end

    # When the client is initialized, use this default behavior
    failsafe def hget
      # put your desired default/failsafe behavior here
    end

    failsafe def hset
      # You've got this figured out by now
    end
  end
end

```

The idea here is that **either** A: the expected `redis_client` with the type `Redis` is passed in and is checked for the required methods to be present (arity would already get checked by the compiler, right?) or B: the `Library::MyClass` is initialized without the client being passed in, and the compiler would use the `failsafe` class definition in it’s place as if it were passed in (skipping the `required_methods` check, ideally).

Something like this would allow shard maintainers to create a public interface that allows for drop-in replacement shards. I really think if we aren’t going to have something equivalent to a Ruby Gems for Crystal, we should have something that allows decoupling our libraries. It doesn’t have to be this exact syntax, but something enabling this type of behavior would be AMAZING.

---

<div class="post-metadata">

**Author:** ![rob](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rob/32/1409_2.png) [@rob](https://forum.crystal-lang.org/u/rob)\
**Post date:** [July 6, 2022, 8:24pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/5 "2022-07-06T20:24:46Z")

</div>

I’ll have to read the wikipedia article you linked sitting next to my compsci bookshelf 😉

I think this is what you mean, translated to this specific example:

```crystal
##### in my library
module Library
  module RedisInterface
    abstract def del(key)
    abstract def expire(key, seconds)
    abstract def flushall
    abstract def hget(key, field)
    abstract def hgetall(key)
    abstract def hincrby(key, field, increment)
    abstract def hset(key, field, value)
    abstract def keys(pattern)
    abstract def llen(key)
    abstract def lpush(key, value)
    abstract def lrem(key, count, value)
    abstract def rpoplpush(source, destination)
    abstract def ttl(key)
    abstract def zadd(key, score, value)
    abstract def zcount(key, min, max)
    abstract def zrangebyscore(key, min, max)
    abstract def zrem(key, value)
  end

  class Configuration
    property redis_connection : Library::RedisInterface?

    def redis_connection! : Library::RedisInterface
      redis_connection.not_nil!
    end
  end
end

##### in an implementation of my library
require "redis"
require "library/redis_interface"

# Monkeypatch in my named type, so Library can match on it and the compiler
# knows that all of the methods I'm going to use are defined.
class Redis
  include Library::RedisInterface
end

```

Do I have that right? If so, that’s at least a path forward. It feels like there may be some gotchas there. I’m unsure how far down the road of typing I can get in that interface. I’m left wondering what happens with named method parameters, too.

---

<div class="post-metadata">

**Author:** ![Blacksmoke16](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/blacksmoke16/32/1241_2.png) [@Blacksmoke16](https://forum.crystal-lang.org/u/Blacksmoke16)\
**Post date:** [July 6, 2022, 10:12pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/6 "2022-07-06T22:12:56Z")

</div>

> [@crimson-knight](#):
>
> But, that would mean we need a tool like Ruby Gems but for Crystal (which I’d love, but that’s a bigger hurdle to overcome I think).

Is that not just `shards`? Granted I’m not super familiar with Ruby, but my assumption was it’s just their package manager for libs.

> [@crimson-knight](#):
>
> Right now it’s basically impossible to write a shard that accepts any library that would meet any interoperability requirements.

Not impossible, but deff requires extra boilerplate to properly handle. I.e. all the adapter logic. It also wouldn’t be out of the question to require the user to implement their own adapter/type based on some interface. This would remove the burden from you, but would ofc make it a touch harder to use. This could be a viable solution for cases where there are too many possible libraries you want to integrate with, or if they want to use a lib that you don’t provide a built in solution for.

> [@crimson-knight](#):
>
> I have this wild idea that we could have a class definition with the required methods and some minimal default behavior in the event the required class isn’t passed in.

Is this not the same thing as your library providing a default implementation of an interface also defined by your library? E.g.

```auto
def initialize(redis_client : Redis? = nil)
  @redis_client = redis_client || DefaultRedisClient.new
end

```

> [@crimson-knight](#):
>
> You’ve probably felt this pain at some point though, being a framework maintainer in Crystal and being required to know exactly what all your dependencies are means you have a very tightly coupled system.

> [@crimson-knight](#):
>
> So, you’re right on the money that the community would need a single “redis” shard that did everything we need.

My main point of friction is [[RFC] Standardized Abstractions](https://forum.crystal-lang.org/t/rfc-standardized-abstractions/4414/1), which could be solved right now by adding more interface modules to the stdlib, or even having a dedicated shard to contain them, and using them in certain places. This would not require any new compiler/language features.

As I mentioned in that other thread, PHP handles this via [PHP Standards Recommendations - PHP-FIG](https://www.php-fig.org/psr/), which define interfaces/requirements for various common libraries/implementations, such as eventing, caching, and HTTP clients, factories, and messages. They also have it somewhat easier for Redis in particular since there is only 1 implementation, the C extension.

Depending on the use case, it may not actually be a problem if we do something similar to PSR in that there is a more general `CacheInterface` that libraries could depend on, with each Redis implementation, implementing that interface. Granted this wouldn’t solve the issue of easily switching Redis implementations, but it might if you’re just using it power some more generic caching layer.

> [@rob](#):
>
> Do I have that right?

Close, but instead of monkey patching the external redis library yourself; you would (or require the user to) provide additional types(s) that wrap the external lib implementation to transform its methods to your interface.

Something like:

```auto
class Library::Adapter::RedisA
  include Library:RedisInterface

  def initialize(@redis : External::Redis::A); end

  # Define methods that map the API of the external shard to your API
  # Could also maybe leverage `delegate` if they're the same
  def del(key)
    @redis.delete key
  end

  # ...
end

```

As you pointed out, the main benefit of this is you are totally decoupled from both what underlying redis client you’re using, and insulated from changes within that lib. I.e. you can be as type strict as much as you want, as long as you do the required transformations to make each external redis shard adhere to your interface. E.g. using `.as` as needed or what have you.

The point in all of this is making it so _your_ code only depends on the abstractions/interfaces it defines. Then it become super easy to plug in implementations other than redis if you really wanted as nothing says, unless you want it to be, that your `Library::Config` type depends on redis, but maybe just `ConfigInterface` or `CacheInterface` as i mentioned earlier.

The only gotchas with this approach is 1) the extra boilerplate required in your (or the user’s code), and 2) dealing with breaking changes in the underlying library that affect its public API and by extension your adapter. Tho, in this case you could either do version comparisons with macros, or just define another adapter for that version 🤷.

Also checkout [[RFC] Standardized Abstractions](https://forum.crystal-lang.org/t/rfc-standardized-abstractions/4414/1), as it’s also related to decorating types.

---

<div class="post-metadata">

**Author:** ![mdwagner](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mdwagner/32/1077_2.png) [@mdwagner](https://forum.crystal-lang.org/u/mdwagner)\
**Post date:** [July 6, 2022, 10:30pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/7 "2022-07-06T22:30:05Z")

</div>

Just wanted to add that you can declare types on those abstract methods for the interface:

```crystal
module RedisInstance
  abstract def flushall : Nil
  abstract def zcount(key : String, min : Int32, max : Int32) : Int32
  abstract def hgetall(key) : Array(String)
  # etc.
end
 
class MyClass
  include RedisInstance
 
  def flushall : Nil
    puts "flushed"
  end
 
  def zcount(key : String, min : Int32, max : Int32) : Int32
    1
  end
 
  def hgetall(key) : Array(String)
    [] of String
  end
  
  # etc.
end
 
my_class = MyClass.new
if my_class.is_a?(RedisInstance)
  puts "my_class is a RedisInstance"
else
  puts "my_class is not a RedisInstance"
end
# => my_class is a RedisInstance

```

Also, I second that [RFC](https://forum.crystal-lang.org/t/rfc-standardized-abstractions/4414/1).

---

<div class="post-metadata">

**Author:** ![rob](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rob/32/1409_2.png) [@rob](https://forum.crystal-lang.org/u/rob)\
**Post date:** [July 7, 2022, 1:34am UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/8 "2022-07-07T01:34:05Z")

</div>

> [@Blacksmoke16](#):
>
> Close, but instead of monkey patching the external redis library yourself; you would (or require the user to) provide additional types(s) that wrap the external lib implementation to transform its methods to your interface.

Yeah, that’s what I intended to show happening. The “install instructions” would highlight needing to decorate whatever Redis class with my abstract module as I have done in my example. If there were some more lifting necessary, where some adapter methods were required they can be included there too. For Redis, this is something that’s not really an issue because everything is strings. I think we’re tracking on the same page.

Re Rubygems, I think @crimson-knight is talking about [RubyGems](https://rubygems.org/) the website and central distribution mechanism for gems, but they’re also the folks behind the cli `gem` tool – gems _can_ be installed from git, but the default is to pull them from [rubygems.org](http://rubygems.org). The point being that if there’s a central source then de-facto standards become more obvious. This has been rehashed a dozen times over and over, and over, and over.

---

<div class="post-metadata">

**Author:** ![Blacksmoke16](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/blacksmoke16/32/1241_2.png) [@Blacksmoke16](https://forum.crystal-lang.org/u/Blacksmoke16)\
**Post date:** [July 7, 2022, 4:49pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/9 "2022-07-07T16:49:34Z")

</div>

> [@rob](#):
>
> Yeah, that’s what I intended to show happening. The “install instructions” would highlight needing to decorate whatever Redis class with my abstract module

Perfect yea. Could also provide a default adapter for your library’s preferred Redis lib. That would make getting started a bit easier, while also leaving the extra flexibility.

> [@rob](#):
>
> The point being that if there’s a central source then de-facto standards become more obvious. This has been rehashed a dozen times over and over, and over, and over.

Ahh okay, that makes more sense. Yea I don’t think shards is ever going to go centralized, but I think we have some options to make that easier.

---

<div class="post-metadata">

**Author:** ![jgaskins](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jgaskins/32/2449_2.png) [@jgaskins](https://forum.crystal-lang.org/u/jgaskins)\
**Post date:** [July 7, 2022, 9:39pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/10 "2022-07-07T21:39:51Z")

</div>

> [@Blacksmoke16](#):
>
> > [@crimson-knight](#):
> >
> > But, that would mean we need a tool like Ruby Gems but for Crystal (which I’d love, but that’s a bigger hurdle to overcome I think).
> 
> Is that not just `shards`? Granted I’m not super familiar with Ruby, but my assumption was it’s just their package manager for libs.

It is, but it’s a (mostly) centralized one. If you define a single source for Ruby gems in your project (this is the most common usage), you’ll never have naming collisions like we do with Crystal shards.

---

<div class="post-metadata">

**Author:** ![rob](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rob/32/1409_2.png) [@rob](https://forum.crystal-lang.org/u/rob)\
**Post date:** [July 13, 2022, 5:08pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/11 "2022-07-13T17:08:38Z")

</div>

In Crystal 1.5, [Warn on positional parameter mismatches for abstract def implementations by HertzDevil · Pull Request #11915 · crystal-lang/crystal · GitHub](https://github.com/crystal-lang/crystal/pull/11915) “Warn on positional parameter mismatches for abstract def implementations” means this technique will probably be difficult to implement as I’ve represented it here because not only the method names but also the parameter names will have to be in sync across various implementing libraries. I find that unlikely even in the most rigorous specifications.

I don’t think it’s a deal-breaker though. The same pattern can be used this way:

```crystal
module Library::GaskinsRedisInterface
  abstract def expire(key, seconds)

  # code from Library will call this method:
  # - interactions of types can be cleanly isolated here
  # - eventually this will delegate to the abstract def above
  def library_expire(key, seconds)
    expire key, seconds
  end
end

```

It’s laborious at best, but it’s workable. It provides a place for an adapter pattern if the interfaces are disparate enough as well.

---

<div class="post-metadata">

**Author:** ![Blacksmoke16](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/blacksmoke16/32/1241_2.png) [@Blacksmoke16](https://forum.crystal-lang.org/u/Blacksmoke16)\
**Post date:** [July 13, 2022, 6:24pm UTC](https://forum.crystal-lang.org/t/abstracting-out-adapters-dependency-injection/4762/12 "2022-07-13T18:24:19Z")

</div>

Answered in [[RFC] Standardized Abstractions - #17 by Blacksmoke16](https://forum.crystal-lang.org/t/rfc-standardized-abstractions/4414/17). Posting here for posterity.