There was a recent post on Reddit asking why there wasn’t an LSP for Crystal. This is a copy / paste of what I said in response, repeated for visibility and further discussion.
There are a number of things that make writing a language server for Crystal difficult.
There is no fault tolerant parser, which is necessary to provide functionality with “broken” code (well, there was before the tree sitter got a lot of improvements, at 99.07% of the stdlib parsing without errors now)
Crystal’s complex syntax makes it incredibly difficult to make a fault tolerant parser
There is no semantic analysis done (by the compiler) to methods that aren’t used
A full semantic analysis of the stdlib and all files is required (due to the nature of the language) every time a change is made
Type inference means that the types of variables and parameters depends on how they’re used, requiring semantic analysis of everything to ensure types are accurately captured
Everything is in a global namespace, and analysis relies on one or multiple entrypoints that have to be specified explicitly
Crystalline has been around for a few years and it does the best for what it has, re-using a lot of the semantic analysis that the compiler uses. It works for some people, especially on small to medium sized projects. I’d recommend giving it a try to see how well it works for you.
I’ve been working on a language server since last June, very much far from done but here’s a list of what I have so far:
Basic framework for a language server to be implemented, pulling a lot of inspiration from Crystalline, named larimar
Started doing a lot of research into fault-tolerant parsers and language server architectures (Roslyn, Rust, etc), and even started implementing my own fault-tolerant parser, but this is an immense task that may not be the best approach
Began to put a lot of work into improving ameba, a linter for Crystal that I find very useful and should be used more widely by the community
I’ve integrated both ameba and the tree sitter back into larimar to provide linting, syntax errors, and syntax highlighting in an editor-independent way. It’s not 100% yet for those but it’s a starting point
My current approach that some people may not like, is that to get LSP-like features, methods / parameters / variables need to be explicitly typed. It’s the only way to avoid needing a full semantic analysis to resolve types.
Having spent the past year and a half working on tooling surrounding Crystal, I’ve become extremely familiar with the limitations of the language, and how it’s seemingly designed to be as hard as possible to have good tooling for. I know a lot of people in the community may be dissatisfied or frustrated with the current state of tooling, but the limitations of Crystalline are the limitations of the language itself, and it’s very hard to work around / outside of those without some compromise.
Progress is being made, but it needs to be done carefully and will take time.
Even if you get fault tolerant parsing right, there’s no a lot you can do to offer responsive autocompletion. For that you need to type-check everything and that’s slow.
An alternative is to only offer suggestions that may be possible, but not all of them, which might be good enough and it might also be responsive.
For example, in this code (where | is where the cursor is):
def foo(x)
x.|
end
we know nothing about x. But we still know it’s an Object so we could suggest every possible method in the system, or maybe in this particular case just methods defined in Object to shorten that list.
But then if you have this:
def foo(x)
y = x.abs
y.|
end
We don’t know what’s the type of x but we know it has an abs method. We could search all the types that have abs in them and know y is the return type of those… if the return type is explicit (though in this particular case we could type-check abs because it has no arguments). Then we offer methods of those types.
You kind of need to replicate the logic of the compiler here, where you “kind of” type-check a method with whatever information you have and proceed to the next expression, type-check it, etc., but in a different way than what the compiler does.
If we have a type annotation on an argument:
def foo(x: Foo)
x.|
end
then we can use that to offer methods from Foo or any of its subclasses.
Given the type-less nature of Crystal, this is in my mind the correct LSP server to build for Crystal, not one that offers precise completion (because that’s impossible, and also very, very slow).
For sure. Simple heuristics can give you like 70% of experience of full intellisense and is definitely better than what we have right now. Like I don’t know what irb does, but it’s honestly pretty good, I wish I could get that experience in my ide.
I think I was originally approaching this of requiring types to get autocomplete, but I think you bring up a really great point I haven’t thought of, and a potentially different way of looking at this problem.
In the case of this:
def foo(a)
b = a.|
end
I think all possible methods should be autocompleted, regardless of what foo is called with, if at all. Then whatever methods are called on a, not only restrict the type of b based on the return type of the methods, but also restrict the type of a based on what objects have those methods. So:
def foo(a)
b = a.abs
# Let's pretend this restricts `a` to `Number`
a.|
# ^ autocomplete only methods on `Number`
a.bar
# ^^^ error: method `bar` doesn't exist on `Number`
end
I may go with the more strict need-to-type-things approach to start with, but this does provide a lot of promise for larimar and ameba to do partial / lossy semantic analysis (I intend to re-use the same visitor for both).
I wonder if it’d also be possible to bubble up parameter types from their bodies? Like in the same example:
def foo(a)
b = a.abs
# Let's pretend this restricts `a` to `Number`
end
foo("bar")
# ^^^^^ error: parameter `a` should be `Number`, not `String`
# or
# ^^^^^ error: String does not have method `abs`
Something to explore once more a framework is laid out.
The only issue I’ve had with it is that it slows down over time… I don’t spend much time programming Crystal to invest in debugging this; but it works for my other LSP needs.
If you’re on Reddit — I’m not — please share this there
At first, this made me sad. Then I thought a bit more about it. In practice, it’s probably quite fine, as one tends to type much anyway.
But this would be extra cool.
I’m not sure I agree. My method might only require the argument you pass it answer to [](String) with a String, but if I typehint it to Hash(String, String) I limit your options on what you can pass. No passing it a HTTP::Headers, no matter how convenient it might be.
I think that’s a bit of a special case. In quite many cases it’s entirely possible to easily type most parameters.
And I think that’s generally a good idea because it documents the interface of a method.
Well, maybe in 2.0 then… Anyway, I just wanted to point out that not everybody thinks “just type everything” is all it’s cracked up to be, but I won’t derail the threat further.
But an LSP that can complete on variables with known types, and make an educated guess for those without, would be excellent. Is larimar in a state where it’s worth trying out, or is it still to early?
That is true. There is a middle ground of something closer to type hints rather than type restrictions (see crystal-lang/crystal#15334) that I’ve also been thinking about.
Things are still too early, the only things that exist currently are semantic tokens using the tree sitter and ameba integration. I’ll let people know when there’s something that’s worth testing.