Project-relative `require`?

Help & Support
1

Is there a workaround to make require relative to the project root? I’m thinking something like how Ruby’s Bundler traverses up the directory tree until it finds a Gemfile — we could look for a shard.yml. Maybe there’s a macro I can use instead of require to do that?

I don’t mind doing relative requires in application code, but loading some of those files from my specs isn’t a great experience. From a few directories deep in spec/, going all the way up to the project root and then down the same path under src starts looking like bots probing for directory-traversal vulnerabilities on web servers:

require "../../spec_helper"
require "../../factories/user"
require "../../factories/ap_object"

require "../../../src/services/ap/undo"
require "../../../src/queries/remote_follower"

I think what I’m looking for is something like this:

# Spec files load from project root to distinguish them from src/
require "spec/spec_helper"
require "spec/factories/user"
require "spec/factories/ap_object"

# src/ files don't need the `src/` prefix
require "services/ap/undo"
require "queries/remote_follower"

Ideally, I’d like to do this in code rather than by modifying CRYSTAL_PATH so nobody has to modify their workflows to run specs.

2 Likes
2

I think it would be cool to have that option. One tricky part would probably be that shards like athena/src/components/dependency_injection at master · athena-framework/athena · GitHub which have nested shard.yml may get a little tricky.

One neat thing we have in our webpack is this “alias” setup

.alias({
    "@": "src/js",
    "@components": "src/js/components",
    "@images": "src/images",
    "@mutations": "src/js/apollo/mutations",
    "@pages": "src/js/pages",
    "@queries": "src/js/apollo/queries"
  })

So in our Vue files, we can do import Modal from "@components/Modal". Not to say that Crystal needs an alias config option like this, but if maybe it was just always assumed that requires starting with @ meant “from project root” or whatever would be cool.

I think there was some discussion around this thing before. We actually had a bug in Lucky pre-crystal 1.0 where our specs had the wrong number of ../ in the requires, but yet the specs still worked :joy: thankfully that’s been fixed now, but yeah, when you start nesting super deep like that, it gets confusing ( like this )

1 Like
3

Yeah, I think there was some discussion around this in Lucky.

What I proposed was introducing a __ROOT__ magic constant (or some other name) that would point to the directory where you run the crystal command.

That way in a spec you’d be able to do:

require "#{__ROOT__}/spec/spec_helper"

and that would be the same regardless of where the spec file is.

That said, I only see this useful for spec files, and for apps.

4 Likes
4

Interesting. I didn’t realize anyone was doing that due to the one-shard-per-repo thing.

Ooh, I could work with this. I think I’d wrap a macro around it, though.

macro req(file)
  require "{{__ROOT__}}/{{file.id}}"
end

macro src(file)
  require "{{__ROOT__}}/src/{{file.id}}"
end

# spec/models/user_spec.cr:
req "spec/spec_helper"
src "models/user"
5

It’s not really as it appears. How I migrated Athena to a Monorepo...and you can too goes into things in more detail, but the gist of it is that all development happens within the monorepo, but then changes are synced out to read-only repo mirrors via git subtrees. This allows you to get the developmental benefits of a monorepo, while still adhering to the “one-shard-per-repo” requirement.

In regards to tests, they’re all still invoked on a per component basis. E.g. https://github.com/athena-framework/athena/blob/master/scripts/test.sh. So some sort of __ROOT__ const would still work.

Although I haven’t really run into a need for it too much. In the spec context I just require all my fixtures, helpers, and source code within spec_helper then require that one file in each spec versus the specific thing each test uses on its own.

A pipe dream of mine is having something like import - JavaScript | MDN, versus having require be global. But deff not something we’ll see soon, or ever ha.

EDIT: Which you can kinda replicate with like:

macro import(type)
  private alias {{type.stringify.split("::").last}} = {{type}}
end

import MyApp::SomeNamespace::Things::MyClass

pp MyClass.new

but :person_shrugging:, debatable of its usefulness, at least without proper IDE support and such.

7 Likes
6

That’s a pretty sweet macro, tbh. I didn’t know we could do private alias but it makes a lot of sense. I use private def and private macro a lot — especially in specs.

1 Like
7

useful for spec files and for apps

My first experience about this exact problem was in tests. It’s a little cumbersome to have to manually traverse the directory tree to import a global test helper. But test files don’t really move that often so it’s not a pain point that scales with a codebase.

Needing string interpolation in the require statement to solve the issue makes me feel like it isn’t an elegant developer friendly solution — it increases the boilerplate rather than reducing it. Thats easily overcome with a macro… but in this case we are talking about a file which would in all likelihood contain that macro — you wouldn’t want to declare that macro right before you use it!

The ruby paradigm of the configurable require paths isn’t used a ton, but does present a polished developer interface.

8

if use this in ruby, it should be Dir.pwd, i guess we add this to Crystal is more better?

you always can set a variable somewhere in project config place, and load it before start app.

ROOT = Dir.pwd # set this some where in initializer

# use ::ROOT here
require "#{::ROOT}/spec/spec_helper"
9

I’m not sure if using some variant of pwd would be a good idea for this purpose. It assumes that you run the crystal compiler from the project directory. If you cd into a spec folder for example and build test programs directly from there, paths would immediately brake.
The beauty of file-relative require paths is that they always work (as long as the file tree itself is not alterered).

10

If you cd into a spec folder for example and build test programs directly from there, paths would immediately brake

Is there any test suite support this? i am a TDD guy, but, never see a suite support this.

i saw a case, in ruby, when run test use rake in a gem, you can run it in whatever folder, but, when you print Dir.pwd in test file, you still get gem root folder, i guess it use same folder as Rakefile.

11

If you cd into a spec folder then nothing will work at all, because CRYSTAL_PATH has lib in it, and that’s at the root directory.

I think it’s pretty fair to assume everyone runs their spec with the console in the project’s root directory.

1 Like
12

Nothing from Ruby applies here. require works at compile-time, so you can’t use anything that happens at runtime (like Dir.pwd) for this.

13

That’s a good point.

It only matters when you actually use something from lib. And the default CRYSTAL_PATH can be changed :person_shrugging:
Probably not enough of a counter argument.

1 Like
14

After some experimentation, I came up with this (examples in gist): Project-local requires in Crystal · GitHub

15

I thought Discourse embedded gist links. Oops. Here’s the macro code to save a click:

# Experiment with macros to allow for project-local requires
# https://forum.crystal-lang.org/t/project-relative-require/4617

{% begin %}
  ROOT = "{{system("pwd").strip.id}}"
{% end %}
macro spec(file)
  macro load_spec
    \{% path = __DIR__.gsub(%r{\A{{ROOT.id}}}, "").gsub(%r{[^/]+}, "..").id %}
    require "\{{path[1..]}}/spec/{{file.id}}"
  end
  load_spec
end

macro src(file)
  macro load_src
    \{% path = __DIR__.gsub(%r{\A{{ROOT.id}}}, "").gsub(%r{[^/]+}, "..").id %}
    require "\{{path[1..]}}/src/{{file.id}}"
  end
  load_src
end
2 Likes
16

Great if that works for you.

But what is the use case for requiring spec files that way?

17

Same as it was in the first post in the thread. This:

src “services/foo/bar”

… is more expressive, easier to type, and less confusing than this:

require “../../../src/services/foo/bar”

And I figured if I did it for src, I could do it for spec to load supporting files for my specs, as well, like factories to prepare state for integration tests.

18

Hi, can anyone help on me for a question about macro?

we use the src macro which write by @jgaskins

# src/config/require.cr

{% begin %}
  ROOT = "{{system("pwd").strip.id}}"
  ROOT1 = `pwd`.strip
{% end %}

  macro src(file)
    macro load_src
      puts ROOT  # <= both ROOT and ROOT1 output same result.
      puts ROOT1 
      \{% path = __DIR__.gsub(%r{\A{{ROOT.id}}}, "").gsub(%r{[^/]+}, "..").id %}
      require "\{{path[1..]}}/src/{{file.id}}"
    end
    load_src
  end

Above code works as expected.

My question is, what is the different with ROOT and ROOT1 ?
In fact, when i try to p! it, ROOT and ROOT1 have same type(String), and same value.

but, when i change {{ROOT.id}} with {{ROOT1.id}} in above source code, get following error.

 ╰─ $ cry spec/spec_helper.cr 
Showing last frame. Use --error-trace for full trace.

There was a problem expanding macro 'load_src'

Code in macro 'src'

 7 | load_src
     ^
Called macro defined in macro 'src'

 1 | macro load_src

Which expanded to:

 > 2 |       puts ROOT1
 > 3 |       
 > 4 |       require "../../../../../../src/config/env"
             ^
Error: can't find file '../../../../../../src/config/env' relative to '/home/common/Study/Crystal/test_macro/spec/spec_helper.cr'

Can anyone can help on why this happen?

# spec/spec_helper.cr

require "../src/config/require"

src "config/env"
19

I guess when i try puts ROOT, ROOT1, i get Runtime value, right?

so, is there a way to get the compile time value of ROOT and ROOT1 ?

20

hello zw963

In your above code, the {% begin %} / {% end %} will generate code. it generates something like

 ROOT = "/your/current/pwd"
 ROOT1 = `pwd`.strip

And does system("pwd") at compile time because it is between {{ }}. So you see that ROOT will carry your current pwd at compile time, whereas ROOT1 will execute pwd at runtime.

However ROOT1 is a constant. That mean if it assigned to a literal value (e.g 42, [1, "foo", "bar"]) the compiler could know its value. That why constants could be acceded in macro land. However here, the compiler could only know that the value of ROOT1 is a call (to the method strip with receiver the call `pwd`).
When you use {{ ROOT1.id }} in your gsub, it’s expended to (`pwd`).strip:

{% path = __DIR__.gsub(%r{\A(`pwd`).strip}, "").gsub(%r{[^/]+}, "..").id %}

which so doesn’t work as expected.

To display a value at compile time:

{% puts ROOT %}
{% puts ROOT1 %}
{% puts ROOT1.id %}
2 Likes