Shard development_dependencies issues

Hi,

Some shards related to databases requires many backends as development dependencies, it makes sense for people developing those shards because they need to test it against many different backennds, however people using those shards usually only cares about one or two backends.

tl;dr; I don’t want to add mysql and sqlite library as build dependency of my project if they wont be even linked to the final executable.

A possible solution is to have two kinds of developer dependencies.

  • First one meant to be used when the shard is used as a library.
  • Seconds one meant to be used when the shard is not used as a library.

To do this in a backwards compatibility way shards would keep development_dependencies as is and add:

  • lib_development_dependencies, triggered when using the shard as a library.
  • app_development_dependencies, triggered when using the shard as an application.

So people developing micrate or some ORM could have mysql, postgres, etc as app_development_dependencies and this wouldn’t affect people using these shards, that will only need to add the backend they use as dependency.

What do you guys think?

Trying to wrap my head around this…

Even if there was a way to skip installing development dependencies that you don’t need: When you install micrate as a dependency, its postinstall hook builds it as an app and thus requires the app dependencies anyway.
So IMO the main issue here is postinstall forcing a build on you for an app that you don’t even need (c.f. Shards' `postinstall` considered harmful).
And if micrate (as a library) doesn’t have hard dependencies on all database shards, it shouldn’t declare them as such.

I’m doubting the dependency system should be bother with this complexity.
If a shard has significantly different dependencies as an app, compared to being used as a lib, it might make sense to have two shards then, with different dependencies.

3 Likes

The same problem happen if the shard doesn’t use postinstall

Maybe what’s missing in shards to cover this use case is the concept of optional dependency. Everything declared there is a hard dependency.

:thinking:

Development dependencies are optimal, but are take all or nothing, so not really useful for this use case.

What if shards ignore the development_dependencies of shards in lib/? Ignoring the fact that this is a breaking change

This is already the case, isn’t it?
Say you have shard foo declared as a dependency. shards install will install foo and foo’s dependencies, but not foo’s development_dependencies.

Concrete example: If you install micrate as a dependency, you’ll get all the db shards because they’re in dependencies. But you won’t get spectator which is development dependency of micrate.

I feel the issue come from the micrate? development_dependencies should be optional, this shards shouldn’t force user install those unnecessary shards, instead, it should add some plugin code to make
those package optional.

Optional dependencies work in that way that you just declare them as actual dependencies of the shard that actually has a hard require on that dependency.

Almost, the problem starts with your application has development_dependencies and you want to install just “yours” development dependencies, not all libraries development dependencies.

Sorry, I don’t get that. development_dependencies are only installed for the main shard (i.e. where you run shards install), not for any of its dependencies.

1 Like

So my whole assumption was wrong the entire time :laughing: and we can forget everything I wrote in this thread.

And I see that micrate has the backend dependencies not in development_dependencies like I thought.

1 Like