Why people leaving crystal?

Community
81

Readability is definitely a subjective subject. While there isn’t a lot that can really be done to address that directly. There are some things that could be done to treat the underlying reason(s) of why is macro code usually so hard to read? As someone who works with them a lot, the main reason is that you can’t reuse your macro code. If you want to do some complex piece logic twice, you have to duplicate everything. IMO I still think [RFC] Macro Defs - Methods in Macro land · Issue #8835 · crystal-lang/crystal · GitHub would go a long way.

Some other issues that could go a long way in promoting a better user experience, and/or architectural patterns.

Make custom compile time errors in macros report the correct node:

Plus some some stuff related to annotations (is better than the mutable constant approach most libs used in the past):

EDIT: Another pattern I realized you can do that helped a lot is if your macro does a bunch of logic before emitting any code, you can include all of that in a single {% ... %}. E.g. go from:

{% paths = {} of Nil => Nil %}

{% defaults = globals[:defaults] %}

{% for a in m.args.reject &.default_value.is_a? Nop %}
  {% defaults[a.name.stringify] = a.default_value %}
{% end %}

{% if (value = route_def[:defaults]) != nil %}
  {% for k, v in value %}
    {% defaults[k] = v %}
  {% end %}
{% end %}

to

{%
  paths = {} of Nil => Nil
  defaults = {} of Nil => Nil

  m
    .args
    .reject(&.default_value.is_a?(Nop))
    .each { |a| defaults[a.name.stringify] = a.default_value }

  if ann_defaults = route_def[:defaults]
    ann_defaults.each { |k, v| defaults[k] = v }
  end
%}

And you end up with much more normal Crystal looking code. Mainly since you can use normal #each methods and dont need all the {% ... %} everywhere.

ref: Routing component integration by Blacksmoke16 · Pull Request #141 · athena-framework/athena · GitHub

9 Likes
82

I do have a suggestion for Macros but I’m not sure how plausible it would be. Originally when I talked to some of the devs at Code Camp I asked about why macros are their own thing kind of separate from Crystal and they mentioned a big source of problem was compilation time going through the roof when recursively compiling code. Can crystal i be leveraged to solve that problem?

83

If I could replace vuejs (or solidjs, or react, etc.) with a WASM-first SPA crystal library and the build size was competitive with solidjs (competitive with vuejs at the minimum), and a sane FFI for the occasional hard-to-want-to-clone npm library, there’s a lot of things I’d leave typescript for.

A helloworld crystal wasm (missing the ability to fully replace vuejs/solidjs) I compiled earlier this year had these results:

-rw-r--r-- 1 forge forge 3.2K Jul 31 10:30 main.js
-rwxr-xr-x 1 forge forge  53K Jul 31 10:30 main.wasm

I’d accept the consequences of a 53k minimum build size if it were possible to do the full SPA framework (replace vue/solidjs) in 100% crystal.

3 Likes
84

Could you please provide more details about this? from my point of view, i agree Blacksmoke16, macro Readability is definitely a subjective subject, and what is the more readable macro which can achieve same effort as current?

Macros also have weird limitations that are not immediately apparent

Yes, this is a issue, my understand,some method missing when write macro always depressed.

1 Like
85

Oh, that was a loooong time ago. I think we had that talk with waj around CurryOn, I think 2015.

We had the idea that macros would work on AST nodes, just like they do now, except that instead of interpreting the code we would compile the macros and execute them.

That is, if you had something like:

macro getter(x)
  def {{x.id}}
    @{{x.id}}
  end
end

Then when you invoked it like this:

getter :foo

we’d create a program that had getter receive a SymbolLiteral, compile that, execute it and paste the result into the program. We originally wanted SymbolLiteral to actually be the SymbolLiteral that was parsed by the program. But what if SymbolLiteral used getter in its definition? (it does now, or, well, it uses the property macro) Then we’d first need to compile a program for property, but that requires AST nodes to be defined, but those could use macros, etc.

I think that’s the recursion that we didn’t know how to solve.

Well, one thing would be to not use any macros in the compiler. But that would be a pointless limitation.

Another idea we had was that instead of passing the actual SymbolLiteral parsed by the current program, we’d transform those into separate types that matched the memory layout but didn’t use macros. But it felt like a hack, and we’d have to constantly keep the two types in sync.

In the end we decided to interpret things instead of compiling them. The downside is that macros end up not being Crystal code.

I think there are some ideas to add macro methods, etc. But they all have the problem that they don’t let you use the full Crystal language. Maybe with the interpreter that would be possible now, I don’t know.

My guess is that we’d need to come up with a different macro syntax and behavior, similar to how Elixir works, where macros are code that generate AST nodes, and not a template language. But everything has its pros and cons.

Anyway, I think that was the recursion we talked about. But I also think that that recursion problem would still be there with the interpreter, if we’d use the types from the compiler for macros.

5 Likes
86

Thinking a bit more about Elixir, macros work there very well because each module is compiled separately. To use a macro from a module you first need to compile it. That gives you a clear dependency graph.

The problem with Crystal is that such graph doesn’t exist and it’s impossible to do.

I said this a thousand times already, but to allow Crystal to further evolve we must rethink some of it to allow clear dependencies between files or types, and allow modular compilation.

11 Likes
87

To say they are totally unreadable was too far and honestly my mistake. There are just a lot of cases of it being hard to read in community projects. For me it’s more the weird and seemingly random limits macros will have, oddities with things like verbatim, errors and other problems not being presented in a weird way, and sometimes, though rare, the occasional Crystal compiler crash. I still think that Macros in general are the way to go through, they are the best compromise to what Ruby offers and they are genuinely powerful enough that most use cases are pretty straight forward, I’d just like to see their inner workings move closer to Crystal. In fact there are clear areas where it contributes to readability, especially with really repetitive tasks, but once you try to get too cute with it things can be difficult to read and debug. We also do have a lot more tools now to help with debugging macros but we could use better error presentation and handling for them overall. This does lead to a lot of confusion especially for new people.

4 Likes
88

while the top level execution is very convenience, it worries me that some library can run arbitrary code in the runtime without your control. We are relying too much on the trustworthiness of our fellow Crystal developers. I just hope it won’t bite our asses in the future like in nodejs community.

4 Likes
89

I don’t see how you can have community-provided libraries without arbitrary code execution. Be that at install time, compile time or run time (you can’t get rid of this last one).

4 Likes
90

eh, my post was not about compilation time vs runtime, it was runtime top level execution.

In crystal, when you require some file then all lines of it, plus all lines of the files that file is requiring can be executed, even if it lies deep inside a class instance method. So just by requiring some files, you are allowing it to do whatever it wants.

So like I said, while this is very convenience (you can do a lot of neat thing like generating constants on the fly) it poses some huge security risks.

By restricting the execution scope, like in languages like Elixir, it is easier to audit because you can track the calling path.
And it will enable modular compilation and a lot of other compilation optimization, too.

I guess it will be not the same language that we come to love any more, but that’s the trade off I’m willing to make. There is just no other language that not only a pleasing to the eye, has a rich and useful standard library and still have superb performance like Crystal. The only missing part is the compilation time, which I really really hope it can be solved one day.

5 Likes
91

I couldn’t agree more.
I’m a huge fan of Crystal. Was and will continue to be.
The only reason why I had to switch away was the lack of libraries.
Just to name some (that are specific to my case) …

  • LDAP
  • Json schema validation

I would agree to the LSP comment; we definitely have room for improvement.

100% of the professional developers in my network use Linux for production, and either Linux or mac for local development. So from my limited perspective, the issue is not with limited windows support.

3 Likes
92

I recently entered this community and authored a lib for pdfs as a c binding. That was a challenging and fun experience. Coming from Go I like the expressive syntax. The runtime and c call overhead makes it a good choice for writing new things on top of already given c libraries. Now coming for the reasons why I fell it is hard to stay productive as take it for serious projects:

  • Managing different c lib dependies ist difficult or not documented well (versions/variations on different os)
  • The compile times of bigger projects with frameworks like lucky and the like are insane - 30-60s on Linux native on a decent i7,32ram,ssd was killing fun/efficiency in minutes
  • vs code / lsp is broken as already mentioned and lags far behind experiences like with Go

I understand that these are difficult to tackle issues and I what to thank the authors for such a great language. I really see great work done here. To get more mainstream adoption and people using this language for professional use the above three points would need to develop significantly.

8 Likes
93

We use LDAP at work (although we connect through PHP). I bet it’s possible to create crystal c bindings from php-src/ldap.c at master · php/php-src · GitHub and whatnot. Although, probably will take some dev time. Now that I think of it, that’s actually a good idea/suggestion. For example, thank the heavens we have the crystal-db shard, otherwise I still might be using nodejs :P.

I can see how that will make crystal more appealing to even more users

2 Likes
94

I haven’t followed the entire thread but I have successfully used this shard (https://github.com/spider-gazelle/crystal-ldap) with great success when I needed to access our LDAP directory in production.

6 Likes
95

Oh, that’s awesome! @kefahi looks like there’s a shard already haha.

A lot of the magic is done here and here. Guess no c binding is needed, as it’s just a TCP connection and parsing the result? Basically conforming to RFC 4511: Lightweight Directory Access Protocol (LDAP): The Protocol?

4 Likes
96

Fixing inconsistencies in language, changing counterintuitive designs, and making language beautiful can appeal to a lot of obsessive-compulsive and perfectionists