My 2022 Crystal Wishlist

Community
1

It has been a momentous 2021 for Crystal.

Version 1.0 was released in March 2021

followed by the first (virtual) Crystal Conference in July 2021.

In the spirit of the Holiday Season,

Here’s my 2022 Crystal Wish List, in no particular order.

  1. At least one more full time (paid) Dev

Adding @straight-shoota as a fulltime dev greatly speeded up Crystal development, and improvement. More fulltime devs will enable more aspects of the language to be focused on simultaneously. More devs means the need for more $$$ to pay them. Hopefully that will happen.

  1. Faster|Incremental Compiling

Compiling speed seems to be always raised by people coming from other languages. It’s a known issue, if for no other reason than marketing.

  1. Better Multi-threading

Crystal has a very young multi-threading model|implementation primarily based on fibers, which is geared more for concurrency than parallelism.

I would love to see it become as good as Rust for true parallel processing. Rust has the crate Rayon that provides most of that for applications. I would love for Crystal to provide comparable features and utility.

  1. Smaller Binaries

I consistently see this raised in the Rust forums too.

  1. More|Better Documentation

You never can have enough (good) documentation, especially as the language and features are growing|changing. This has been brought up over and over, so it’s nothing new as a desirable element of the Crystal ecosystem.

I also suggest creating a Crystal Youtube Channel as a single focal point for people to see|submit video tutorials, examples, project showcases, etc.

  1. Marketing Strategy

You can have the best xyz, but if nobody knows about it, or how to use it, and how it can make their lives easier, better, more productive, it doesn’t matter.

This directly ties into the documentation issue too.

  1. Another Crystal Conference

The July conference was great! Thanks to everyone involved in pulling it off. Of course the Ruby world has lots of conferences around the world which its community look forward to. This also fits into marketing and general education.

  1. Google Summer of Code (GSoC)

I’ve seen this discussed before. Maybe it can be done in 2022.

  1. Web Assembly (WASM)

It’s emerging, it’s sexy, it’s useful, it’s part of the future.
Crystal can (should) be a big player in this field.

  1. Killer App

When you’re good at everything, people overlook you. When you’re known for at least one thing, more people pay attention. It just seems that’s the way it is. Allot of people still don’t know there’s a difference|separation between Rails and Ruby.

This is my short list. What’s yours!

15 Likes
2

Well, in order of most wanted

  1. Full windows support
    and I can guarantee the project will get more funding once it is available on windows
  2. Better documentation
    Just like you said. Personally I prefer written items over videos but a better youtube presence wouldn’t hurt. One thing that that is clearly improvable is to write documentation that does not rely on users knowing ruby at all, and likewise, to expect non-ruby users.
  3. Faster or more tuneable compilation
    once again like you said
6 Likes
3

RE Documentation: has anyone thought of creating something like rubydoc.info to auto-build/cache the crystal docs for shards given their github/gitlab user/repo name? That way the shard authors wouldn’t have to host their own documentation and it would all be in one place.

2 Likes
4

Can just use https://pages.github.com/ with a GH Action, e.g. oq/deployment.yml at master · Blacksmoke16/oq · GitHub.

4 Likes
5
3 Likes
6

Integrations with popular 3rd party services. So much of modern development is integrating with other applications in the ecosystem.

3 Likes
7

(in my opinion) the reason why ruby didn’t become what python now is, is sinple windows support and easy gui toolkits. So, thats my number 1 wish.

3 Likes
8

My only wish in 2022 is for the type system and some core semantics to be formalized and finalized, so that the compiler can be developed against a truly formal specification and compiler non-developers don’t have to guess what happens behind the scenes (in cases like this for example).

4 Likes
9

Happy new year!

My only wish for Crystal in 2022 is a better compilation time.

Bonus:

  • Better support of Crystal in VScode
  • Better documentation
  • WASM
2 Likes
10

I can’t agree more, but that’s a wish for 2022-2026 (the time span of a PhD…).

11

We have a similar list going in Lucky. My Lucky "Wishlist" · Discussion #1629 · luckyframework/lucky · GitHub

Personally, I want to see more experimentation done with DSLs specifically in the web sphere. Most of what has been written was written by people coming from Ruby and mimicking libraries there, but I think there’s so much left unexplored. Athena is a good example where it’s using Crystal’s annotations in a really clean way that no other popular framework has done and that’s because the creator was inspired by PHP rather than Ruby. I think Kotlin is a source of inspiration for me even though I have used it very little. I’m looking at libraries like GitHub - JetBrains/Exposed: Kotlin SQL Framework and wondering if it’s do-able in Crystal.

3 Likes
12

As you could probably guess, I’d like some improvements/thoughts around macros/annotations:

Having all these would be huge in regards to the power, ease of use, and readability of macros/annotations.

2 Likes
13

The only thing in my wishlist is to think about Crystal 2.0 as a new language that retains the original spirit of the language but changes enough things to be able to implement modular compilation. I have to sit down and think about what this means, and my wish is for other core team members to also try to think about that.

To achieve modular compilation, all of these would need to happen:

  • Instead of require merely loading files, change it, or introduce an import keyword to specify what you want to import. If you don’t import something, it’s not globally available (unlike the current behavior)
  • Mandatory types in method arguments and return types
  • Disallow reopening types, but allow something similar by having extension methods (similar to C#)
  • Type restrictions in generic type arguments
  • Macros can no longer introspect the entire program. They can still introspect types

At least, the idea is that a file can be looked at and type-checked by looking at itself and its dependencies, but without having to always have a main program.

That said, I’m not sure if all of the above is entirely possible without making the language lose its essence.

13 Likes
14

In the original list there is a Crystal Workshop 2022, and I’ve been thinking about adding spaces to discuss these kind of topics.

4 Likes
15

Did you just say “be like many other languages” and “remove some of the main reasons people use crystal”?

Nitpicking import - my least objectionable change

Working in C# requires sprinkling ~2-5 additional usings in most files. I don’t see how this helps me as a programmer other than resolving namespace collisions (which shouldn’t happen).

using/import only says which modules/classes a file uses. Which can be gathered by parsing the file and made efficient by caching the result. ← Correct me if I’m wrong.

Sooooo what real benefit did import provide other than repetitive strain injury at the top of every file?

Tone: My critique isn’t directed at you personally but at import and extra work in general.

All others on your wishlist make the top of my wish not list, except for macros which I don’t know enough about.

16

You could try concurrent.cr (self plug), which is similar to rayon

Example:

array.parallel.select { ... }.map { ... }.serial.sum

Or more complicated:

uris.parallel.map(fibers: 50) { |uri|
  # at most 50 parallel http requests
  http_get(uri)
}.map(fibers: 10) { |data|
  # at most 10 parallel db requests performing lookups or other validation
  A::ORM.new(data)
}.batch(1000) { |batch|
  # Back to a single fiber
  # Save 1000 records in a single transaction
  A::ORM.transaction do
    batch.each &.save!
  end
}.wait

It’s a little more (and less?) than rayon and includes parallel channel processing.

Order is not preserved.

Compile with -Dpreview_mt and the fibers run on multiple threads.

2 Likes
17

Yeah, that’s a good point :sweat_smile:

I always have this conflict… like, I really like all the nice features Crystal has, but they make doing modular or incremental compilation impossible. So Crystal use cases are limited to small and medium projects. Large projects will take an awful amount of time to compile, and there’s nothing you can do about that. And I’m not sure that’s good.

That said, when Ruby started more than 20 years ago, apparently it was pretty slow. Then machines became faster and now the performance is pretty good. And I heard the new Mac M1 compiles Crystal code like 5 times faster than the old Mac, so maybe in the future compile times won’t be a problem anymore. No idea!

Well, using/import tells the programmer which file they need to look in case they don’t understand what Foo refers to. It limits the search scope. It also tells the compiler a file’s dependencies, so if any dependency changes, it can know which other files need to be recompiled. With Crystal’s require or Ruby’s require that’s pretty much impossible.

Without having an import system, incremental or modular compilation is impossible to do.

3 Likes
18

Please don’t sacrifice type inference for compile speed. Yes, it takes a bit. but so does building a chair. compared to the time you spend sitting on it, a bit of extra time in manufacturing pales in comparison.

There really is a lot of crying about the compile speed, but please stop a moment and consider if that is maybe a small price to pay for what crystal brings to the table.

6 Likes
19

it is a huge price to pay for me actually.

The more code you add to the project, the more it hurts you back by the compilation time.

the feedback loop becomes longer and longer, and the productivity just drops significantly.

also, the current crystal is way too lax it’s hard to trace the side effects.

20

I’m not sure that’s 100% true.

Rather than have a goal of 100% incremental compilation, use a goal of faster compilation times utilizing incremental compilation when possible or easy.

How could this be done?

Foreach function
  does this function only call primitive operations? Y: precompile
  does this function only call methods that are precompilable? Y: precompile

  cache:
    precompiled status
    precompiled blob
    source or ast of nonprecompiled portion
    list of files defining this method/class and macros used

Solving reopening classes:

On compiler start:
  Scan search paths for new/deleted/modified files
    foreach file
      Invalidate caches for classes/methods modified

Solving lack of import

The compiler maintains a list of file dependencies foreach class/method
When parsing a file, every namespace referenced is equivalent to an import statement
When the file changes:
  reparse and invalidate all referenced precompiled methods/classes recursively
  recompute the implied imports

Now the problems are:

  1. Maintaining/invaliding the various caches
  2. Compiling the remaining parts that can’t be precompiled
1 Like