Language Server Plans

Community
1

Plans for a new Crystal language server implementation have been brewing for a while, and with the growing interest in Crystal editor and tooling support, I feel that this a great time to share those plans and get more community engagement/feedback/support for it.

The Crystal Tree Sitter

The official tree-sitter implementation is still in the works but is already a huge step in the right direction for editor and tooling support. For those that don’t know or need a refresher, tree-sitter is a blazingly fast incremental parsing library that can provide detailed information about the source it’s parsing (which in this case, is Crystal). This already gives us a head-start as the complexities of the language can be accurately expressed in a simple grammar rather than having to re-implement those stages in Crystal. Plus, the added benefit of having first-class support in many editors gives enthusiastic Crystalists the opportunity to take advantage of tree-sitter while it’s still being developed, instead of having to wait for stable releases of other tools.

Library bindings

@hugopl’s crystal-tree-sitter already bridges the gap between Crystal and tree-sitter, and with our joint efforts this will only improve over time. Thanks for your work so far!

Putting things together

The main idea for this language server is to leverage the power of tree-sitter to provide general syntactical analysis. From a feature standpoint, that means:

  • Incredibly fast symbol generation and lookups
  • Retroactive project indexing and analysis
  • Goto definition support
  • (potential) Basic static analysis
  • (potential) Integrated formatting

and a few other useful features. From there we can delve into semantic analysis. As mentioned in the Incremental compilation exploration post, this would likely require projects to be strictly typed simply because of the complexity of type inference in Crystal. The information can then be cached and retroactively updated rather than having to be re-evaluated on every language server event. Feature-wise, that means:

  • Context-based goto definition support
  • Context hover information
  • Method and type completion

and a lot more other features. Not to mention, the possibilities with direct file integrations like with shard.yml, spec integrations and debugger implementations would now be possible via the language server. There is truly a lot to look forward to!

Timeline

All these great things mentioned will of course take time to implement, and at present the language server is not even remotely close to being able to handle any of these features — this is something that will take months to complete. There are also things not listed that may or may not be implemented depending on feasibility and complexity, but those can be discussed in this forum. For now, all I can say is that the future of Crystal editor and tooling support is a bright one!

12 Likes
2

Do repositories exist already for this? Would be nice if you could edit them into your post. Also, maybe a separate discord channel + link to it?

3

There is a private repository that will become publicly available once we have a stable project structure and plans which I will post once they are available. I’m not sure if we could get a separate channel in the Discord server so I might end up making a separate Discord server.

3 Likes
4

This shard is in a toy state, I was literally playing with a possible API for it to use it in a editor for syntax highlighting and later for some basic code models for a very basic auto-complete/code navigation feature.

I thought before about use tree-sitter to create language servers, but doing this means that the language server will have some limitations by design unless you want to basically rewrite the compiler using the tree-sitter AST.

2 Likes
5

I figured as much heh. Myself and a few others were planning to contribute to it in time if that’s something you’re interested in, otherwise we would just maintain a fork of the shard.

That is indeed the plan. One of the limitations of the compiler (besides being slow for language server standards) is the lack of fault tolerance—that being it stops at the first error encountered rather than continuing to check the rest of the code—which would be severely limiting as errors from one file would directly affect language server capabilities/features throughout the whole project. Using tree-sitter to manage the syntax side of things allows the language server to handle errors in the project while still being able to use features in the rest of the project.

3 Likes
6

Contributions are welcome, forks too. The language server is a very good use case to model an crystalized API for tree-sitter.

2 Likes
7

Emacs support tree-sitter since 29.

there is a tree-sitter project available for languages prebuild binary which works with Emacs, It supports 85 languages, but unfortunately, Crystal is not among them.we did really poor on this point.

 ╰─ $ ls
ada.so*         cmake.so*       erlang.so*         groovy.so*        jsonnet.so*   ocaml-interface.so*  scala.so*       vhdl.so*
agda.so*        commonlisp.so*  fennel.so*         haskell.so*       julia.so*     ocaml.so*            scheme.so*      xml.so*
arduino.so*     cpp.so*         fish.so*           hcl.so*           kotlin.so*    pascal.so*           smithy.so*      yaml.so*
asm.so*         css.so*         fluent.so*         heex.so*          latex.so*     perl.so*             solidity.so*    zig.so*
astro.so*       csv.so*         fortran.so*        hlsl.so*          lua.so*       pgn.so*              sql.so*         
bash.so*        d.so*           gdscript.so*       html.so*          make.so*      php.so*              swift.so*       
beancount.so*   dart.so*        git-rebase.so*     jai.so*           markdown.so*  prisma.so*           tcl.so*         
bibtex.so*      dockerfile.so*  gitattributes.so*  janet-simple.so*  matlab.so*    python.so*           toml.so*        
BUNDLE-VERSION  dtd.so*         gitcommit.so*      java.so*          mermaid.so*   r.so*                tsx.so*         
c-sharp.so*     elisp.so*       gitignore.so*      javascript.so*    meson.so*     rst.so*              typescript.so*  
c.so*           elixir.so*      glsl.so*           jsdoc.so*         nix.so*       ruby.so*             typst.so*       
clojure.so*     elm.so*         go.so*             json.so*          noir.so*      rust.so*             verilog.so*

I just started using tree-sitter and don’t understand how to add it yet, but i guess
tree-sitter-ruby can be a good start.

@hugopl , Can i know how is your project different from tree-sitter-ruby?

@Devonte Does your project refer to the tree-sitter-ruby above? If not, could tree-sitter-crystal benefit from your project?

8

The Crystal tree-sitter was using Ruby’s tree-sitter as a reference for a small period of time but it wasn’t a good source, largely because the parser is inscrutable as discovered in the Discord server, but also because the languages are very different syntactically.

The language server and tree-sitter are separate projects. The language server is largely dependent on the tree-sitter so there won’t be much development on the language server until the tree-sitter is stable.

1 Like
9

Mine is just a incomplete binding to libtreesitter, not a tree-sitter language implementation, this is why it’s called crystal-tree-sitter and not tree-sitter-crystal.