# RFC: Link configuration

**URL:** <https://forum.crystal-lang.org/t/rfc-link-configuration/546>\
**Category:** Crystal Contrib\
**Created:** [March 13, 2019, 8:16pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546 "2019-03-13T20:16:35Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [March 13, 2019, 8:16pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/1 "2019-03-13T20:16:35Z")

</div>

Right now the given a `@[Link("foo" ... )]` statement the compiler will do the best it can to find the library to link.

That process is basically for posix environments that for each `foo` library:

1. If `pkg-config` is installed it will ask to it using `pkg-config foo --libs [--static]`
2. Search `libfoo.a` within `/usr/lib`, `/usr/local/lib`

For simple use cases, this is enough. But there are a couple of limitations:

1. It’s hard for the end user to link to a custom build of the library, for example with debug information.
2. If a shard requires a library the user has no opinion whether the library can be linked statically or not.
3. In OSX it is possible to link to a static library in some cases but that requires the full path to `libfoo.a` file.

We don’t want to lose the simplicity of the build process that involves linking, that’s for sure.

The question is how we can enable customization of the linking and have nice defaults that are populated by the compiler and also probably by the shards.

At the end of the day having some lookups of link configuration files at the: project, shard, user, compiler level seems a good idea to me.

Using the foo at `@[Link("foo")]` as a key for accessing that configuration seems simple.

Maybe when building a project by the first time a `.crystal-link.yml` could be generated or expanded with the missing entries. Or a separate step can also provide that expansion directly.

The current arguments to the `LinkAnnotation` could be used to populate this defaults. As long as there is no other `.crystal-link.yml` at project, shard, user, compiler that states a value.

The link file could provide directly the flags to use by the linker

```nohighlight
version: 1
libs:
  foo: /path/to/libfoo.a
  bar: libbar.a -L/usr/local/lib

```

Or could eventually support more features like running a bash/script to detect path/version of a specific library.

Or if no library is found it will be able to emit something in the link config file.

A more simple alternative would be to use convention over configuration. An environment variable per library like `CRYSTAL_LINK_FOO`, if defined, could state the link command to use, if not keep the current behavior. Although it is simple it leaves to the user the burden of managing all those settings.

---

<div class="post-metadata">

**Author:** ![asterite](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/asterite/32/60_2.png) [@asterite](https://forum.crystal-lang.org/u/asterite)\
**Post date:** [March 13, 2019, 8:39pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/2 "2019-03-13T20:39:35Z")

</div>

I think for a file like this:

```crystal
@[Link("foo")]
@[Link("maybe something else")]
lib LibFoo
end

```

the compiler could introduce a new flag, `--link` that would look like this:

```auto
crystal build foo.cr --link "LibFoo=..."

```

and then the compiler would not use the `@[Link]` declarations on top of that lib but instead would read from the command line. I think that `...` that I put there could have the same syntax as what you put inside `@[Link(...)]` and the compiler could simply put that inside `@[Link(...)]`, parse it and apply the same logic.

That way you can configure things based on lib names.

Then, on top of that, we can think of a way to write all of those link options somewhere, but probably putting them in a Makefile should be enough for now.

---

<div class="post-metadata">

**Author:** ![jhass](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jhass/32/1779_2.png) [@jhass](https://forum.crystal-lang.org/u/jhass)\
**Post date:** [March 14, 2019, 9:12am UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/3 "2019-03-14T09:12:21Z")

</div>

How do other languages supporting binding declarations handle this?

---

<div class="post-metadata">

**Author:** ![RX14](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rx14/32/633_2.png) [@RX14](https://forum.crystal-lang.org/u/RX14)\
**Post date:** [March 14, 2019, 1:20pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/4 "2019-03-14T13:20:51Z")

</div>

Having a file for this seems super overkill. Traditionally it’s the user’s job to set up the linking environment for the linker (`LD_LIBRARY_PATH`, `PKG_CONFIG_PATH`, etc.) and the compiler is agnostic. But I’d accept some easier to use linking overrides through a `--link-override` flag or `CRYSTAL_LINK_OVERRIDES` env var, since some configurations make this hard.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [March 14, 2019, 5:34pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/5 "2019-03-14T17:34:08Z")

</div>

Right now the compiler does not check for `LIBRARY_PATH` at all. The only path that will be used are the one informed by pkg-config or `/usr/lib`, `/usr/local/lib`.

`pkg-config` is an optional dependency for crystal. The user can create a custom `.pc` file to link specific file and even force a static version of the library. But then we end up forcing the user to actually write a “configuration file”.

Using a single environment variable to list directories to look up for lib & static libs based on name should be the simplest for the user. But that still does not cover how we can choose when building whether we prefer static or shared linking. Yet is far more simple than writing a custom `.pc` file. I think this can be used for libraries embedded by the compiler like pcre, gc, compiler-rt, etc. And letting the wrapper script that calls the compiler define the path as the embedded ones.

In a more flexible scenario I like @asterite proposal of `--link` to override _all_ `@[Link]` of a given `lib`.

But having a configuration file seems more controllable IMO. But it would be good to support the former two aspects (env variable + link option) even if we don’t end up supporting a configuration file.

---

<div class="post-metadata">

**Author:** ![wontruefree](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/wontruefree/32/1467_2.png) [@wontruefree](https://forum.crystal-lang.org/u/wontruefree)\
**Post date:** [March 14, 2019, 6:55pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/6 "2019-03-14T18:55:35Z")

</div>

What about adding the libraries to `shards.yml` and having the compiler search for any linkable libraries inside the `./lib` or `./lib/links` directory?

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [March 14, 2019, 6:58pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/7 "2019-03-14T18:58:24Z")

</div>

I think that boils down with the same story as how to customize/override shards declaration per project basis. Which is something [I would still like to see](https://github.com/crystal-lang/shards/issues/105).

---

<div class="post-metadata">

**Author:** ![RX14](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/rx14/32/633_2.png) [@RX14](https://forum.crystal-lang.org/u/RX14)\
**Post date:** [March 14, 2019, 10:01pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/8 "2019-03-14T22:01:29Z")

</div>

We do use `LD_LIBRARY_PATH` because when crystal fails to find a library it falls back to `-lfoo`, and the linker looks at `LD_LIBRARY_PATH`.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [March 19, 2019, 6:36pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/9 "2019-03-19T18:36:05Z")

</div>

Implemented the env var configuration for lookup part at [#7562](https://github.com/crystal-lang/crystal/pull/7562)

---

<div class="post-metadata">

**Author:** ![ysbaddaden](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/ysbaddaden/32/2252_2.png) [@ysbaddaden](https://forum.crystal-lang.org/u/ysbaddaden)\
**Post date:** [March 20, 2019, 5:54pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/10 "2019-03-20T17:54:08Z")

</div>

I think I don’t understand the issue, because I never thought linking was a problem:

- If I want embedded static libraries to be used in a crystal distribution, I specify `export LIBRARY_PATH=/path/to/crystal/embedded/lib:$LIBRARY_PATH` in the crystal shell wrapper script so the linker will find them —this achieves the same as #7562.

- If I want to link a specific library installation —for example OpenSSL— if `--link-flags` isn’t enough or the library provides a pkgconfig file, I tweak the `LIBRARY_PATH`, `LD_LIBRARY_PATH` and/or `PKG_CONFIG_PATH` environment variables; For example:

- If I want to link to a system library in a shard I use `@Link("foo")` (see stdlib).

- If I want to link a locally built static library in a shard, I use `@Link(ldflags: "#{ __DIR__ }/ext/libfoo.a")` (see [scrypt](https://github.com/ysbaddaden/scrypt-crystal/blob/master/src/lib_scrypt.cr#L1) or libcrystal in stdlib);

- If I wanted to link to a locally build shared library, I’d use `@Link(ldflags: "-L#{ __DIR__ }/ext -lfoo")` but using a locally built shared library is a bad idea in a shard, because executing the program requires `LD_LIBRARY_PATH` to be exported to actually use the library.

---

<div class="post-metadata">

**Author:** ![asterite](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/asterite/32/60_2.png) [@asterite](https://forum.crystal-lang.org/u/asterite)\
**Post date:** [March 20, 2019, 6:09pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/11 "2019-03-20T18:09:55Z")

</div>

@ysbaddaden I think the problem we are trying to solve here is that someone wrote `@[Link("foo")]` in a shard, which you don’t own, but you want to use that shard but link it to something else, maybe using `ldflags: ...` and a particular object file.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [March 20, 2019, 6:20pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/12 "2019-03-20T18:20:19Z")

</div>

Given a `@Link("foo", static: true)` The current behavior is

- `pkg-config #{libname} --libs --static`
- `pkg-config #{libname} --libs`
- look up `lib#{libname}.a` in `/usr/lib`, `/usr/local/lib`

The user can’t change the library linked without changing the source code or writing a `.pc` and installing/configuring pkg-config.

It is not blocking. But is a way to improve I think the current behavior.

Using `LIBRARY_PATH` will not override the result of pkg-config so, if pkg-config and the library are installed I think is not enough to:

> [@ysbaddaden](#):
>
> If I want embedded static libraries to be used in a crystal distribution, I specify `export LIBRARY_PATH=/path/to/crystal/embedded/lib:$LIBRARY_PATH` in the crystal shell wrapper script so the linker will find them —this achieves the same as #7562.

---

<div class="post-metadata">

**Author:** ![vladfaust](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/vladfaust/32/48_2.png) [@vladfaust](https://forum.crystal-lang.org/u/vladfaust)\
**Post date:** [March 20, 2019, 10:30pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/13 "2019-03-20T22:30:19Z")

</div>

[That](https://github.com/veelenga/lua.cr/issues/30) issue may be related to the topic?

---

<div class="post-metadata">

**Author:** ![Didactic.Drunk](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Didactic.Drunk](https://forum.crystal-lang.org/u/Didactic.Drunk)\
**Post date:** [June 26, 2019, 6:53am UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/14 "2019-06-26T06:53:18Z")

</div>

Perhaps slightly different but what do you do if the library name doesn’t match the pkg-config name?

An example is libsodium:

```auto
lib/libsodium.23.dylib
lib/libsodium.a
lib/libsodium.dylib
lib/libsodium.lalib/pkgconfig/libsodium.pc

```

```auto
# Linking is -lsodium
pkg-config libsodium --modversion
1.0.18

```

If anyone would like to look further, Travis won’t link with a custom path even with LD\_LIBRARY\_PATH.  
[https://github.com/didactic-drunk/cox](https://github.com/didactic-drunk/cox)

I’m posting this here because I’m thoroughly confused about the linking process and how I’m supposed to proceed. This is a one off custom build for testing. I can’t use /usr/local with caching because other files are there which generates a \> 450mb cache file.

The type of thing I’m trying to do can’t be that uncommon. Install 1-2 libraries under a custom path and build an application with shards. The process should be simple and painless.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [June 26, 2019, 12:48pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/15 "2019-06-26T12:48:15Z")

</div>

@Didactic.Drunk, I guess you want to link to `libsodium.a`. For linking agains a custom path there are two options.

1. Prepend `lib/libsodium.lalib/pkgconfig/` to `PKG_CONFIG_PATH`, or
2. Prepend `lib/` to `CRYSTAL_LIBRARY_PATH`. This var is used in the `-L` option of the linker.  
2.1. If this fails, try using `@[Link("sodium", static: true)]` instead of `@[Link("sodium")]` (while keeping `lib/` in `CRYSTAL_LIBRARY_PATH`)

Ref: [https://crystal-lang.org/api/0.29.0/Link.html](https://crystal-lang.org/api/0.29.0/Link.html)

---

<div class="post-metadata">

**Author:** ![Didactic.Drunk](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Didactic.Drunk](https://forum.crystal-lang.org/u/Didactic.Drunk)\
**Post date:** [June 26, 2019, 6:26pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/16 "2019-06-26T18:26:30Z")

</div>

Got it. Thank you.

---

<div class="post-metadata">

**Author:** ![Didactic.Drunk](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Didactic.Drunk](https://forum.crystal-lang.org/u/Didactic.Drunk)\
**Post date:** [June 26, 2019, 6:37pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/17 "2019-06-26T18:37:21Z")

</div>

Is there a way to set environment variables as part of a build? How an I set `PKG_CONFIG_PATH` or `CRYSTAL_LIBRARY_PATH` as part of a build or when running `crystal spec`?

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [June 26, 2019, 6:49pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/18 "2019-06-26T18:49:07Z")

</div>

`VAR=val command` should do it.

---

<div class="post-metadata">

**Author:** ![Didactic.Drunk](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@Didactic.Drunk](https://forum.crystal-lang.org/u/Didactic.Drunk)\
**Post date:** [June 26, 2019, 9:43pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/19 "2019-06-26T21:43:43Z")

</div>

Is there a way to do it automatically as part of the build process or when running `crystal spec ; crystal foo ; crystal bar` where it doesn’t need to be specified on the command line or before running the command or as a wrapper script?

Is there some way to do it in a macro before pkg-config is run?

```auto
{{ setenv PKG_CONFIG_PATH `./find_package.sh` }}

```

I’m searching for the most recent package and building a newer one locally as linux packages are often frozen for a particular release. Old packages often won’t work. Forcing the user to manually compile a package and set environment variables before running `crystal foo` is unworkable.

---

<div class="post-metadata">

**Author:** ![bcardiff](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/bcardiff/32/3_2.png) [@bcardiff](https://forum.crystal-lang.org/u/bcardiff)\
**Post date:** [June 26, 2019, 10:16pm UTC](https://forum.crystal-lang.org/t/rfc-link-configuration/546/20 "2019-06-26T22:16:52Z")

</div>

The current option is to either have a Makefile or a shell script that will export the variables you want.

There is no current solution on behalf crystal/shard to install library dependencies in a uniform way, hence neither is a way to find them.

The beginning of this thread was about having a configuration file for the linked options, that instead of been computed directly, could be generated if missing. Allowing the user to override it.

[Next page](https://forum.crystal-lang.org/t/rfc-link-configuration/546.md?page=2)