Shards' `postinstall` considered harmful

Crystal Contrib
101

This is what I do in my apps that use other shards, defining the tools as targets:

name: REDACTED
version: 0.3.4
targets:
  drift:
    main: lib/drift/src/cli.cr
  app-cli:
    main: src/cli.cr
dependencies:
  drift:
    github: luislavena/drift
    version: "~> 0.3.2"

I like the proposal of targets under dependencies, as it will allow me to decide which targets I would like to get build part of installing them, and is something I can control.

Adding a few more cents to the conversation (by now we should be around 100€) :laughing:

Cheers,

4 Likes
102

I did say it was a proof of concept, not a complete solution. It can be iterated on.

shards knows what platform it runs on. It can install whichever type of script is needed for that platform.

1 Like
103

@luislavena this is such a smart idea :hushed:

104

That’s why MSYS2 exists :thinking:

105

This is something I worked on about a year ago with a Shards-like CLI, the concept is quite straightforward:

scripts:
  postinstall@windows: del /q /s .\build
  postinstall@linux: rm -rf build/

Platforms are specified following @ which can be group targets (“linux”, “windows”, etc.) or triple targets (“x86_64-linux-gnu”, …) for only those platforms. Triple targets have higher precedence than group targets, which in turn have a higher precedence than scripts with no target.

Unfortunately, due to time constraints, I wasn’t able to develop the app as much as I wanted to (hoping to get more time later this year) but the concept worked well for the shards it was tested on (one being a Lua binding shard which can now reliably install lib dependencies on Linux and Windows thanks to this).

3 Likes
106

With no replacement is probably a bit harsh. Now, Shards is still in v0 so we can break whatever :smiling_imp:

:confused:

107

Indeed. But then you need MSYS2 in Windows to use Crystal.

108

This wouldn’t solve the initial use case that impulsed the postinstall feature: build a vendored library that may be seldom packaged or that needs patches. For example my 3 uses-cases:

  • libcmark: used to be seldom packaged (no longer true) and there’s the markd shard which makes it not very interesting to keep (time to archive);
  • libscrypt: not packaged anywhere;
  • Immix GC: it’s written in C with Crystal bindings.

Lazy me thinks it would be nice to have --postinstall=scrypt has I would have nothing to do. Now, it would be the same to document how to install the library, possibly provide a couple makefiles for posix and windows.

Last but not least, I could just port the crypto_scrypt function to pure Crystal. I started to port my GC to pure Crystal with nanolib already (I’m bored with C).

109

You need a C compiler to use Crystal too. I would trest them in the same category.

3 Likes
110

I see them on different levels.
Building a program is a more specific tasks from setting up the development environment.

Also you “only” need a C compiler (which just acts as a linker driver) in order to build (link) an executable.
Various other tasks, particularly related to development, work perfectly fine without a full build environment. Also executing a Crystal program in the interpreter or building an object file for cross-compilation doesn’t need a C compiler.

111

I found this topic to be a really difficult issue.

There are two main ways to solve a problem. One is an idealistic approach, where we define the perfect solution and work backward to determine the necessary actions. The other is a realistic approach, where we predict future outcomes based on the current situation and intervene to make things better.

From the second perspective, removing postinstall completely would be very difficult unless there is a strong commitment. At the very least, I do not expect the Crystal community to reach a full agreement to remove postinstall.

If postinstall is removed, users will likely look for alternatives.
As Sol.vin pointed out, one possible solution is to use macros like {% run some_crystal.cr %} to execute scripts at arbitrary times.
This could result in a more chaotic situation compared to using postinstall.

In the open-source world, many people use Crystal-based tools without being interested in the language itself. For them, downloading and compiling source code is just a simple task, and they do not care about Crystal. For example, if I need to use a Perl tool, I do not try to understand Perl—I just want the tool to work. Manually checking dependencies and running commands to set up binaries is something I want to avoid if possible.

These “lazy users,” who only want to use the tool but do not engage with the Crystal community, probably exist in large numbers. They may be invisible, but they are important for the growth of Crystal’s ecosystem.

I think postinstall is like a “trash bin.” If you have a trash bin, garbage is collected in one place. If you remove the bin, trash will spread everywhere.

However, this does not mean that the total amount of trash increases.
If postinstall is really a problem, removing it and observing what happens could be an option. It may be a pessimistic view, but I believe nothing major will happen. Instead, the problems will either move to different places or become more scattered. Overall, I do not think the situation will improve or worsen significantly.

Unfortunately, I do not know what the best solution is :face_with_head_bandage:
(Translation from Japanese by ChatGPT)

2 Likes
112

I would compare MSYS2 to Termux; you don’t have a native C compiler or package manager available to adb shell, but it is never too much to ask people to install Termux and use Shards from there.

It might even be preferrable to offer a GUI installer that installs MinGW-w64 Crystal + MSYS2, similar to RubyInstaller, for those who don’t want to spend time setting up the latter alone first.

1 Like
113

:+1:

I bet that for most Crystal users, postinstall is never been an issue.
It’s only a problem in the minds of some very senior developers. :sweat_smile:

2 Likes