Crystalline - LSP server for Crystal

Community
1

Hi everyone :wave: ,

I am currently writing Crystalline, a Language Server Protocol implementation (think Scry) for Crystal that provides limited language features (like go-to, autocompletion, syntax and semantic checking).

I’m still working on it, and I’m planning to improve some parts of it. I began using it for my own projects for a few days now and I find it quite helpful, so I’m sharing it here in case people were missing these IDE features a much as I did :slight_smile:.

Small disclaimer: Crystalline does not aim to be as fast - or as feature complete - as other LSP because it would involve a massive amount of effort to write incremental semantic analysis, a fault tolerant parser or other tools needed. Despite that it should be in a fairly stable state - meaning it should not crash or report wrong information. But do not expect it to work in all cases, it will not report anything when type information is missing.

11 Likes
2

I just updated VS Code plugin with the updated dependencies to work with Crystalline.

I hope we can include Crystalline to crystal-lang-tools organisation if you @elbywan want and other organisation members agree (@bcardiff?)

3 Likes
3

I tried compiling it but it never produced a binary or any meaningful error. Just says…

ian@computer:~/Downloads/crystalline$ sudo crystal build ./src/crystalline.cr -o ./bin/crystalline --release --no-debug --progress -Dpreview_mt Killed] [0/1] Codegen (bc+obj)
4

I’m on a Mac. I have a problem with llvm and libxml2.
@elbywan can you help @sol.vin? Have you experience with Windows?

5

This is great!

Could you give a high level overview of how it works? For example what happens when you ask for autocompletion.

6

I’m wondering if it would not be a bit too soon, I would rather wait to see if other people find it useful. It also lacks a lot of polish (incoplete features, no binaries or brew install, undocumented code, no specs).

These are crystal compiler dependencies, which Crystalline uses under the hood.

Regarding LLVM, can you check these instructions?

For libxml, there is a llvm 10 bug on macos when checking system dependencies.

❯ llvm-config --system-libs                                                                                                                                                                                           14:05:22
-lm -lz -lcurses -llibxml2.tbd

(-llibxml2.tbd is wrong)

A workaround is to add a symbolic link:

ln -s /usr/lib/libxml2.2.dylib /usr/local/lib/liblibxml2.tbd.dylib

It is not ideal, but until llvm produces a fix I’m afraid this is required.
More info here.

Well sorry I cannot give more insight on that issue, but as Crystalline embarks the crystal compiler it does require a large amount of RAM to compile, and it takes a while.

Killed] [0/1] Codegen (bc+obj)

Did you kill it yourself? Because if so, it looks like it was still compiling.

In all cases, I’m planning to provide compiled binaries in the future.

Hmm sorry no, I’m not a Window user (except for playing games :video_game:).

7

Well it is quite simple actually. I’m mostly relying on what you guys did already, the Crystal parser & semantic analysis to produce a typed AST and I extract meaningful information using visitors to display that in the editor.

Of course, I try to cache the result as most as possible - when possible, and be a bit clever on how to process requests in parallel, in threads isolated from the language server I/O loop and use some locks to not overload the server.

For instance it will not trigger a full scale analysis when you just hover or go to a definition every time. It will just reuse the last analysis result.

It will also not run 15 compiler tasks at the same time if you ask for autocompletion 15 times after typing a character, it will wait for the pending analysis to complete and run another one afterwards to process the latest updates. And it will not block in the meantime if the user just wants to use the formatter.

There really is a lot of room for improvements, but for small to medium scale projects with a decent computer it seems to be working fine for me.

I don’t have a lot of spare time unfortunately these days, so I won’t be able to write down more than that. The first step would be to document the code and refine the features.

5 Likes
8

Worked for me. Thanks.

1 Like
9

Hi,
Have you interest in using https://github.com/crystal-lang-tools/lsp ? if so, me too, as this repository seems dead, I keep a fork on https://github.com/hugopl/lsp with my patches… and I’m using it to write a LSP client for an editor, Tijolo.

1 Like
10

Hey @hugopl,

I tried using it at first but I had issues because:

  • it was relying on an old version of the standard
  • some json structures conflict when deserializing
  • there is only type definitions in the shard - I needed an up-to-date server

So for Crystalline I wrote the LSP part from scratch alongside a generic server loop implementation. I was planning to extract this part as a shard but as I implemented only the messages I needed and skipped specs so we’ll see in the future!

11

Oh thank you! It’s great!
It is better to uninstall Scry or Crystalline comes as a complement?

1 Like
12

Awesome! :+1: Just write in the README that LLVM 8 is needed (it is not possible to compile with LLVM 10).

1 Like
13

Thanks! :smile:

It’s either one or the other.

1 Like
14

Thanks! :+1:

I don’t really know about other OSes, but on macos it is technically possible to compile with LLVM10 even though there is a bug in the current minor version.

I posted the workaround earlier:

It is not obvious though, and you’re right to highlight the fact that I should definitely write something in the readme :wink:.

15

OK thanks

16

Yep, I’m aware of the first two issues you said, the third one isn’t really an issue because the shard idea is to have only the type definitions decoupled from any client/server implementation… to avoid the boring task of replicating them from the spec… I’m using it because it saved some of my time and as I use only a tiny part of LSP, the structures conflicts aren’t a problem (yet) for me.

I saw you use partial text-sync on Crystaline, this is very good… scry uses full text sync, i.e. ask the client to send the whole file on every change, something really slow and non-sense.

1 Like
17

Yes I understand the idea, and I was actually not talking about a full server implementation.

That was a bit obscure, but when I was talking about “server” I was referring to a class orchestrating the whole protocol (requests / notifications / responses / replies / errors), dispatching the messages to an external entity in a non blocking way when needed and replying to clients. So basically the I/O loop, but without any concrete action taken when a message is processed.

The actual implementation could then be delegated to a controller class separate from the shard.

This is what I tried to do in the /lsp part of Crystalline.

18

Install on Ubuntu 18.04 (WIP).

It’s safer to use the same (or closest) LLVM version that Crystal compiler. When writing this message:

crystal version
Crystal 0.35.1 [5999ae29b] (2020-06-19)

LLVM: 8.0.0
Default target: x86_64-unknown-linux-gnu

So LLVM 8 (but the ./llvm.sh official script support v9 and +, I have a compil error)

git clone https://github.com/elbywan/crystalline
cd ./crystalline

# Install llvm
sudo apt-get update
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 9
# whereis llvm-config-9 (v9 for me), when located add the path to env
export LLVM_CONFIG=/usr/bin/llvm-config-9

# build Crystalline
mkdir ./bin
crystal build ./src/crystalline.cr  -o ./bin/crystalline --release --no-debug --progress  -Dpreview_mt

Then make it executable chmod +x crystalline, then add crystalline to a bin path (executable), like mv ./crystalline $HOME/bin/crystalline or /usr/local/bin or /usr/bin

But I have an error during the Crystalline build. I will test again on the next Crystal version.

19

Hello!

This project is a very good news!

As a contributor on scry (for various reasons I stopped doing crystal for now though…) you’re doing it right!
I’m happy that someone is taking over this subject that way!!

1 Like
20

I wonder if crystalline could be renamed to something else… just because of the fact that now that I have it in my path, autocompleting crys stops at crystal without a space, where previously it was the only binary I had matching that.

Is it needed to put crystalline in the path at all if it’s needed to be configured in vscode?