[Mini Review] Giving up on Crystal

1

For the past several months i have been writing Crystal code for a micro service system with the usual web toys. Over this time i have banged my head on issue plenty of times.

Yesterday i ran again into the enforced Type system. After hours and being a idiot, a simple solution was found. And yet, today when trying to make a more clean code… guess who came back to bit me in the behind. Yes, mr Type System again.

    class Template

        @template : String
        @action : String

       # @string_values : Hash(String, ?) = {} of String => ?

        def initialize( @template : String, @action : String )
        end

        def setString(data_name : String, data : String)
            @string_values[data_name] = data
        end

        def setGeneric(data_name : String, data )
            @string_values[data_name] = data
        end

        def export( )
            hash = {
               "_option_" => { "template" => @template },
               "_target_" => { @action => { "data" => @string_values } }
            }

            return hash
        end

    end

Take this code as a example ( just a example ). The moment you initialize a hash, your stuck with the type that are defined. Even worse is that at times, you do not even know what types can be available. For instance namedTuples from DB rows or different Tuple amounts etc… So this code will always trigger compiler errors, no matter how many combinations you try.

Something as simple in dynamic languages as feeding a mixed types, becomes a huge mess that requires way more knowledge the deeper you go into Crystal.

Being honest. Crystal its Generics are not generics. They are simply a redirected type inference system. You are still required to provide a type, be it on class creation, on output or … Crystal does not have a real generic that can truly bypass the Type Check or does a lazy type check on the provided data. Or override a hash type after creation.

The example above is just that. A Snapshot of a bigger issue. Over time i noticed that you end up constantly writing code, that involves dealing with the type system for very little gain. At times i know that the types are correct and yet, trying to push this information in arrays or hash or move it around becomes so much about the types. Arrrg

On other fronts:

IDE:

Even basic things like a IDE are a big issue. You like a Jetbrain product? Too bad because the only Crystal plugin is syntax + really bad performance issues. Try pasting …

@string_values : Hash(String, String) = {} of String => String

… and enjoy a 4 or 5 second lockup on the IDE. And another 2 or 3 seconds when you move your mouse over it. Or how a simply 200 line module can becomes so slow, that your waiting more on the IDE to responds whenever you press enter, then anything else.

When you get tired off fighting, maybe you think. “Well, its a Linux product, let try Linux”. Here you are installing Mx-Linux, Crystal, VSC and … basic syntax that works. Some code hints but that is about it. You really feel underwhelmed. Most of the Crystal plugins for IDEs are years old and barely supported. I assume most regular users here are VIM users and simply know most of the code from their heads ( ruby background? ). But as somebody coming from a different background, it has been less then stellar experience despite biting my teeth for months.

Compiler:

The compiler can also be a big annoyance. Especially the compile times. I am using Sentry + Parallel to auto build the services. This cuts down on wait times but your still constantly. Save … wait for the auto compile … finally check the result. Sentry cuts down the delay from manually triggering the compile job but it does nothing for the compile times.

Documentation:

A lot of documentation is … what i can described typical for a community project. As in lacking, especially the API side of things. Good for basic understanding the basic language but again, do more then a simple program and you start seeing a lot of missing functions.

Windows and other features:

Its damming to see issues on Github dating back to 2015 mentioning Windows or Multithread support. I am sure it will come one day but for something that Manas in the 2017 “not new year resolution” mentioned for a 2018 goal, it feels like only one guy is really trying to solve that issue. Was the money from the donations not supposed to go to those goals?

Let alone proper IDE support beyond syntax. Let alone expecting actually working IDEs on Windows ( as they are going to rely on Scry compiled to Windows or a Windows Crystal compiler ).

1.0

When i see changes like … HTTP::Multipart to MIME::Multipart, that make me think. Darn glad i am not using Multipart ( yet ) or else this will have broken my code on a Crystal update. Probably with a non-descriptive error because nobody seen fit to include any warning or even a version warning. 0.27.x => This future may be removed or moved in X version. Nowp … just a instant change.

Its one example of many where code can break without any build in warnings. If you want people to treat Crystal for a successful project, pulling this type of constant changes is simply not acceptable. Its a playground currently and that is not a good thing.

Future:

I like Crystal but the more i work in the language… For me it hurts giving up on months of work but to be honest … I rather work with PHP and get stuff done then constantly fighting the language, compile times and other issues. The latest Type issues are simply the last drips in the bucket.

So yes, it has been another day where i ended up at 04h at night trying to fight Crystal to do what I want, not what Crystal wants. And i am fed up and tired of it. I can rewrite this code that took me months probably in a few weeks in PHP and be done with it.

I may way more issue about for Crystal but i am tired and too darn sleepy. So here is my sleep deprived harsh opinion of Crystal. Its a bit of a mess, that might turn into something good. But for now i do not advice it for any big project unless you have the manpower and knowledge.

2 Likes
2

Hey Benny,
thanks for your extensive feedback. It’s much appreciated!
Writing so much text instead of silently running away to PHP seems like an indicator that you might not want to give up on Crystal yet ;) At least I hope so.

Crystal’s static type system is supposed to make it easier for the programmer because the compiler ensures that the value you’re working with is actually of the type you expect it to be. From my experience and other reports, that’s working pretty great and is a big selling point for Crystal. And I have rarely had any bigger issues with it, nor have I heard others complaints. So I’m a bit surprised that you’re condemning the static type system. Without knowing any specifics, I suspect that you’re maybe using it in a wrong way… That’s not meant to accuse you of wrongdoing but we just need better documentation and examples for how to do things.

I don’t fully understand the issue your having with the code you’ve posted as an example. Yes, using container types with specific item types can sometimes be a little bit more effort than in a dynamically typed language. But then it also gives you the safety of knowing that it can’t hold arbitrary values.
I’m not sure what your intention is with the code you posted, but if you need a Hash literal to have a specific type (which can be broader than the actual types in the literal), you can use a typed Hash literal (Hash(String, String | Int32 | Bool){ "foo" => "bar" }). Or, more likely, a Hash isn’t even the best data structure for your problem. Have you considered using a record (or class)?
If you’re interested in suggestions, I’d recommend opening a new thread for this specific issue.

Generics

Generics need a review and improvements (for example explicit covariance and contravariance). We know that and it’s on the roadmap. But it’s not true that you always need to explicitly declare generic type arguments. The compiler is actually smart enough to figure out the appropriate type, if a value of that type is used in the constructor (see https://carc.in/#/r/68uz).

IDE

I don’t think you can blame the language itself for lacking IDE support. Although I don’t find it that lacking, anyway. I don’t use very deep language integrations anyway (and I guess most Crystal developers don’t). For me, the plugins for Sublime Text and VS Code are good enough, they provide some compiler integration and that works even on Windows (with WSL). So actually, for my preferences, this is totally fine.

Compiler

The compiler is not the fastest, and we’re trying to improve compilation times. But honestly, I can’t find this more than a minor nuisance.
What kind of compilation time are you experiencing?

Documentation

I agree that this is a major field needing improvement. As it is with most open source software.

Windows and other features

The money received from donations actually covers only a fraction of the work that has been put into Crystal by Manas and other contributers, and are continually putting into it. We can’t expect to reach all goals with rocket speed like that. But it’s moving forward, that’s the important thing.
And both Windows and MT are not super urgent in a way that they would block other developments of the language or stdlib. So I think it’s fine to say they come when they’re done. It would be great if this is sooner than later. But for that we need more people working on it.

1.0

The relevant part is that we’re currently not at 1.0 and don’t make any promises about backwards compatibility and deprecation cycles until then. This would put to much weight on avoiding breaking changes. And we need breaking changes to effectively improve things.
Yet honestly, the impact of breaking changes in the last Crystal releases has been really minimal. They’re rare and usually simple to solve. Above that, I don’t expect many people using HTTP::Multipart directly, so it’s not a big issue, but gives us a logical, sane API for the future.

So at the end, I am sad to hear that you feel like fighting against Crystal. It’s meant to focus on developer happiness, not frustration. So it’s really bad that it turned out like this for you and I hope we can improve on that experience.
Let us know if we can assist you with any specific issues.

Disclosure: This post is just my personal opinion and not an official statement of the core team.

5 Likes
3

When going down the class or struct route, one can just as well pick Go. One of the main reasons for picking Crystal, was that at the time Crystal looked more flexible with data. Namely, you did not get forced to always use strict code. And while this is partially true, the moment you start with specific code that involves going type less, you run into issues.

Let me expand on my example how easy it is to run into Type issues:

contacts = @db.query_all "SELECT name, age FROM contacts", as: { name: String, age: Int64}
#You end up with a namedtuple, of the type { name: String, age: Int64}

address = @db.query_all "SELECT street, city, postcode FROM address", as: { street: String, city: String, postcode: Int64}
#You end up with a namedtuple, of the type { street: String, city: String, postcode: Int64}

Template.SetGeneric( contacts )
Template.SetGeneric( address )

# To make something like this work, you need to use a normal Tuple with generic data typles like

contacts = @db.query_all "SELECT name, age FROM contacts", as: { String, Int64}
#You end up with a namedtuple, of the type { String, Int64}

address = @db.query_all "SELECT street, city, postcode FROM address", as: { String, String, Int64}
#You end up with a namedtuple, of the type { String, String, Int64}
# And deal in the Template class with it as (Hash(String, Hash ( String | Int32 | Int64, ...) )
# But then it breaks if you have 3, 4, 5 fields

In other words, your just shifting the issue down a other point. And you can bet that if i made a hash with dozens of types and dozen of fields, the compiler is going to be hurting performance wise a lot.

No matter what type you try to do, its hard to impossible to deal with those without resorting to a while different design, as you stated with fixed classes and inheritance. For something as simple to have a structured Json output.

Sure, we can go the Angular way and start building up Classes or Struct in the back and front-end and then keep those in synchronization and have a whole different design. Or we can initiate a hash on every request, and manually add the whole “template, action” each time. Their are ways around it but it forces you as a developer down a few specific rabbit holes. Lets say your structure changes, you do not want to have a 1000 different spots you need to update. Classes / Structs run into the issue of type checks again.

Type checking when you need it. Sure … but you also need the ability to simply say. No … I know what i am doing and i am dumping out Json here. I do not need type checks on this output.

Crystal is a bit like a deep hole. Type, bypass type ( function without type ), and … here come another type check. Sometimes you simply want the type: Any!

This i stated before. Crystal forces the type checking on the class creation. Here is a example:

class AnyWrapper(T)
  def initialize(@object : T)
  end
end

# Type is AnyWrapper(String) without having to declare String anywhere
p typeof(AnyWrapper.new("foo"))

# VS

class AnyWrapper
  def doSomething(@object : T)
  end
end

x =- AnyWrapper.new()
x.doSomething("foo")

# in line 2: can't infer the type parameter T for the generic class AnyWrapper(T). Please provide it explicitly

The second example can not work currently how Crystal is designed. Crystal will only work with generics on: Class initialization, Function output checks on forall and a few other cases. But it has no ability as posted above. But that is a different topic…

And this is why Crystal leaves a bad taste at times. In my case Crystal broke my code on 0.24, and 0.25. Without any clear warnings i may add. Hey, lets move reuse port to a different spot. Great idea!

This is another reason for me to simply give up on Crystal for now. Maybe in a few years but Crystal currently is in the precarious situation. Slow development, major features that the competitors have for years ( MT, Windows, IDEs etc ). The specter of 1.0 can be a red flag for years to come. I remember Nim was supposed to be 1.0 in 2017. We are 2019 and still they keep changing code all over.

Growing compile times… Used to be 2 seconds, now its going 3.5 seconds. Crystal 0.27 actually dropped the compile times down a bit but as code keeps growing, so does the compile times ( Very normal ). But when you make small changes to check the front-end responses, those 1 or 2 seconds even with a auto compiler literally become hell. And this is with a project cut into smaller micro services to reduce code growth.

  • Change something
  • Auto compiler starts. Your mouse moved to your browser …
  • Waiting … yes … compiled. Press F5 or whatever
    … Repeat …

It slowly starts to become frustrating ( and this is on a AMD 1700x ). To the point you start to think. Maybe i will buy a 9900K just so i have have a higher core performance. But that is simply moving the problem again.

In general

I do not want to trash Crystal too much… Its mostly stable and you can tell people put a lot of work in it. But it has still years of development and pain points ahead of it. The Type system was a issue going back to 2016.

My personal tests before really starting this project only scratched the surface of Crystal and now i am seeing more and more of the pain points ( for me ) in the language. And these really influence my productivity to the point that if i do not move away from Crystal now, i will be losing months totally missing my deadline.

I am sure in a few years things will look better with Crystal but for now, i can not jeopardize my company’s schedule. I wish you all the best.

4

What does Template.SetGeneric do?

5

It tries to store “data” in a hash, so later when exporting, you take that hash, combined it with several preset values ( template, action, … ) and badabing, badaboem, you can have a json that your front end can handle.

But as we know, that “data” can be different typle/namedtuples/records with strings, int, one to infinity fields depending what comes out of the database. Your basic run of the mill data going from back-end to front-end for displaying. And we do not need type checking because its going to the typeless front-end anyway.

And yes, manually creating a hash ( forcing a type check on the hash creation ) right after the database data pulling works.


    contacts = @db.query_all "SELECT name, age FROM contacts", as: { String, Int64}

    hash = {
       "_option_" => { "template" => "contacts/template_contact"},
       "_target_" => { "do_contacts" => contacts }
    }

    hash.to_json(response)

but its sloppy way to write the code and means if the structure ever changes, you are going to need to update hundreds or thousands off entries vs one simple update to the template class. Very basic stuff that takes me 5 minutes to write in PHP but results in hours upon hours of head bumping by fighting with the type system in Crystal.

6

It would be awesome if you could share a link to your code, or at least a reduced version of your code to really see the problem. At least I can’t fully understand the problem with all the pieces disconnected.

7

Excuse me, @Benny, why don’t you want to use an ORM in the first place to work with the DB?

8

Its not about “fixing” my code but the issue that the Type checker simply gets in the way for data that can be different on every requests. As such, you are required to handle every request differently and can not use a generic class.

Its a issue that can not be solved without adding some kind of “Any” / real generic system, that can override type checking. Or by redesigned applications to only work in a specific way ( as stated above, with negative consequences ).

Orm’s abstract away at your data and make it harder to problem solve. And a ORM has no use in this specific instance because even any ORM code will run into the type checker. Unless you build a orm system that dynamically builds up every statement with a macro. Fun with compile performance when doing that a few hundred or thousand times :slight_smile:

Just tired of fighting Crystal for simple tasks, that take 5 minutes in other languages but force you to make some Frankenstein monster in Crystal with Macros or trying to bypass the system with creative solutions.

Just close the topic Asterite. I have nothing more to add.

9

Just to throw in a small coin, it took me a while to start thinking in types after years of everything-is-a-Value dynamic languages. If you really need something done fast and dirty it always gets in the way, but I like solving puzzles, and fortunately I write code mostly for fun, so when my mess finally compiles it’s a huge dopamine rush. Having said that, there is definitely a room for improvement, I’ll give a small example.

I’m creating a bunch of different channels and I want to have a global hash with all my Channel(K)s. In Ruby? Easy! In Crystal? :thinking: Hash(T, Hash(Int32, Channel(T)))? After some thinking I’ve realized that I don’t need the parametrized part of Channel to store, patched it to add a close_proc method that returns a closure around Channel.close method and ended up storing that. It feels like a hack though, I think what I wanted to say was Hash(Int32, Closeable) or something, but then there would be a need to factor out the invariant (I hope I use the right term) from all the existing code, which is a huge task.

TL;DR: Crystal can be and frequently is a source of great frustration for a person coming from dynamic background, but such is the nature of compilers with static code analysis.

10

@Benny I think I know the pain you were going through… JSON handling in Crystal forced me to write any_hash.cr shard which takes pretty much anything JSON format can handle as an input, while totally sacrificing type system - which is pretty handy for manipulation and (de)serialization, but awful in terms of reading from it since values are Union of all possible types…

11

I definitely think your rant is more of Compiled/statically typed code vs script code.
You would write C/C++ code you would face the same kind of issues.

Concerning the compile time, I think it’s matter of production pipeline; obviously in scripted language it’s instant so you do small modifications => run the output, while in compile languages you don’t. I used to work in video game industry (before Unity/Unreal Engine) frameworks and I promise you don’t build the whole game every time you make a change. On other hand, you run into way less errors as compiler catch most of them.

Currently working on a webapp with Crystal (Mithril/Typescript in front, PG as database), I can tell that it’s way less productive than if I wrote it in Ruby on Rails, but mostly because I chose a lightweight stack (Kemal).
So compared to Sinatra or ExpressJS, I would say the productivity is almost the same. A bit more of boilerplate, but so far using JSON::Any for simple objects and data mapping for complexes/redundant structure and I’m good.

2 Likes
12

Very sad to see you go, @Benny.

Compilation

I can’t speak about your generic issues. With over several thousand lines of code split between my master and gameserver, I’ve never used generics/templates and never will. So I can’t comment on those specific issues as I really have no idea.

In regards to compile times, I do share your frustration. My compile times are around 3-5 seconds, compared to just 1-2 seconds in the beginning. However, it’s a double-edged sword. I feel like the longer the compile times are, the more I need to focus and have that confidence of writing good code. Why? Because this will allow me to write more instead of writing 1 line, then “compile and test”. That get’s annoying real fast. The more confident the developer is, the more code they can write without doing that. It feels good. This also increases the developer’s confidence level more and more (the more you code without “compiling and testing”).

But on the other hand, it’s frustrating sometimes if you made a small typo and need to “compile and test”. You have to wait. For me, it’s a couple extra seconds in the game lobby waiting to press the “create game” button. It’s a small detriment to workflow speed; but then again, I made the typo, my fault.

Also, I’m on WSL where compile times are actually a detriment compared to native Linux. I’ve added a couple hundred lines of code in the past few months and compile times seemed to have evened out / roughly the same. I found that interesting.

Documentation

I agree, but I wouldn’t say they are necessarily lacking. There are a lot of great examples in the docs I found, and if I didn’t understand how to use a certain method, I’d post it on gitter or reddit and got my answer immediately

Windows and other features

Honestly, I don’t think MT is as big of an issue as people make it out to be. Obviously it would be awesome, however, Nodejs doesn’t have it, and it’s still wildly popular. I don’t think it’s a game changer at all. If anything, single threaded apps can be just as powerful and safer if you scale with redis, or, use a PM2 like management system for Crystal.

1.0

I agree. The unix timestamp change pushed my buttons a bit. Had to search and replace a lot of code to Time.now.to_unix_ms. It took a couple minutes, but now I’m over it, haha. Godot does the same thing sometimes. IMO, it comes with the territory of open source projects and developers must learn to adapt.

However, if stuff starts getting changed where it continuously breaks code, is when i’ll get upset. But I don’t think the core developers would allow that. So I don’t think people should be worried.

2 Likes
13

Time.now.to_unix_ms will break in the next release. Sorry for this. But it’s easy to replace, and actually you shouldn’t have used Time.now when you only need a Unix timestamp. Use Time.utc instead, it avoids time zone lookup when you don’t need a time zone anyway.

2 Likes
14

Confused, can you not just create a data structure that holds all types, then case through it to parse the values?

class Data
    def initialize()
        @data = {"Int" => [] of Int32, "String" => [] of String, "Bool" => [] of Bool}
    end
def load(json : String)
    json = json.strip("{")
    json = json.strip("}")
    puts json
    json.split(",") do |element|
        k, v = element.split(":")
        k = k.strip()
        v = v.strip()
        case v                                       # I don't have to time check but you would have to get the raw type
        when String
            puts "#{v} is a String"
        when Int
            puts "#{v} is an Int"
        when Bool
            puts "#{v} is a Bool"
        end
    end
end
end

sorry new to the language (like 1 day new, lol). I came from python, never really liked ruby, but I wanted a compiled language without semicolons that had OOP constructs. :stuck_out_tongue: stupid go, nim… Seems like they do something similar in the https://github.com/crystal-lang/crystal/blob/master/src/json/any.cr
either way, I think the language looks promising, definitely new; but excited to see where it would go

15

Did you mean to post that in my thread? :D