# Tree-sitter Crystal **URL:** \ **Category:** Community\ **Created:** [July 27, 2021, 6:08pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565 "2021-07-27T18:08:32Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![will](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/will/32/1280_2.png) [@will](https://forum.crystal-lang.org/u/will)\ **Post date:** [July 27, 2021, 6:08pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/1 "2021-07-27T18:08:33Z")
[Tree-sitter](https://tree-sitter.github.io/tree-sitter/) is a parsing library that was just brought into neovim 0.5, and I think some other editors use it too. It’s intended to be used by any editor, so like LSP servers, they can all share a common tool for a given language. It is really cool since it does fast incremental parsing, so as you type it can reparse just the new part of the tree, and it handles errors (such as a partially typed keyword, or whatever). This talk from the author is good [Tree-sitter - a new parsing system for programming tools - Strange Loop](https://www.thestrangeloop.com/2018/tree-sitter---a-new-parsing-system-for-programming-tools.html) By having your editor be able to actually parse the code instead of just doing some complicated regexes, you get much better and faster syntax highlighting. But more than that it starts to enable refactoring tools that deal with the actual AST. I started trying to write one for crystal, but it’s a pretty daunting project, and I know I’ll never be able to finish it myself. If anyone is interested, I’ve put what little I’ve got so far (just symbols) [GitHub - will/tree-sitter-crystal](http://github.com/will/tree-sitter-crystal) and I think something like this would work well rubinus-commit-bit style, where anyone who gets a PR in gets commit. Hopefully eventually it can get in a state where it gets added to the [http://github.com/tree-sitter](http://github.com/tree-sitter) org since that seems to be like the package manager of sorts for the grammars. ---
**Author:** ![kim](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/kim/32/693_2.png) [@kim](https://forum.crystal-lang.org/u/kim)\ **Post date:** [July 28, 2021, 3:05pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/2 "2021-07-28T15:05:48Z")
Amazing! There was another partial implementation I stumbled across recently which may be of interest: [GitHub - keidax/tree-sitter-crystal](https://github.com/keidax/tree-sitter-crystal). ---
**Author:** ![will](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/will/32/1280_2.png) [@will](https://forum.crystal-lang.org/u/will)\ **Post date:** [July 28, 2021, 3:29pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/3 "2021-07-28T15:29:20Z")
Oh that’s cool, I did search around of a bit before starting it, but didn’t find this one. Thanks! ---
**Author:** ![spencerwi](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/spencerwi/32/91_2.png) [@spencerwi](https://forum.crystal-lang.org/u/spencerwi)\ **Post date:** [July 29, 2021, 7:02pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/4 "2021-07-29T19:02:07Z")
I am _very_ interested in this same idea – I also use nvim 0.5, and I’d also love a tree-sitter parser for Crystal. I’m not especially good at tree-sitter (I’ve tried poking at keidax’s grammar linked above recently with no real success). But if I’ve got time and capability, I’ll have to shoot a PR your way with whatever I can get working. ---
**Author:** ![asterite](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/asterite/32/60_2.png) [@asterite](https://forum.crystal-lang.org/u/asterite)\ **Post date:** [July 29, 2021, 7:25pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/5 "2021-07-29T19:25:23Z")
I didn’t read anything about this, nor seen the video/presentation, but… does this have to be implemented in C, and with all that grammar thing? Would it be possible to reuse Crystal’s lexer and parser for this, which are written in Crystal? ---
**Author:** ![will](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/will/32/1280_2.png) [@will](https://forum.crystal-lang.org/u/will)\ **Post date:** [July 29, 2021, 7:51pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/6 "2021-07-29T19:51:15Z")
You could maybe get by with some sort of trick compiling the parser down into a static lib and including that, but it’d be hard. However one of the main features is the recovering from errors, here you can see in ruby I put an error in the middle, but it was able to spot that error, and continue parsing ![image](https://canada1.discourse-cdn.com/flex036/uploads/crystal_lang/original/2X/a/ae95826dbd6f464924a3027474a60f402ff87136.png) I’m not sure if the Crystal parser can do that? ---
**Author:** ![asterite](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/asterite/32/60_2.png) [@asterite](https://forum.crystal-lang.org/u/asterite)\ **Post date:** [July 29, 2021, 8:05pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/7 "2021-07-29T20:05:04Z")
Ah, I see. Yeah, the parser can’t do that, but it could be modified with a flag to allow those scenarios. Though it’s strange… where does the error live? It looks like there’s a `binary` AST node, but it has an `ERROR` right after `left` and before `right`. Be warned that implementing the lexer and parser for Crystal in C is a lot of work! The lexer has about 3200 lines of code, and the parser has like 6000 lines. And Crystal is pretty concise! ---
**Author:** ![will](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/will/32/1280_2.png) [@will](https://forum.crystal-lang.org/u/will)\ **Post date:** [July 29, 2021, 9:03pm UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/8 "2021-07-29T21:03:22Z")
Yeah it’s not going to be trivial, but the ruby one seems to just be 1k lines in the js dsl [tree-sitter-ruby/grammar.js at master · tree-sitter/tree-sitter-ruby · GitHub](https://github.com/tree-sitter/tree-sitter-ruby/blob/master/grammar.js) then another 1k of c++ scanner code [tree-sitter-ruby/scanner.cc at master · tree-sitter/tree-sitter-ruby · GitHub](https://github.com/tree-sitter/tree-sitter-ruby/blob/master/src/scanner.cc) , then some other things that seem generated to me. ---
**Author:** ![vlazar](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/vlazar/32/31_2.png) [@vlazar](https://forum.crystal-lang.org/u/vlazar)\ **Post date:** [July 30, 2021, 4:44am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/9 "2021-07-30T04:44:10Z")
If I understand correctly writing scanners (in C or anything else) is not always required in tree-sitter [Tree-sitter｜Creating Parsers](https://tree-sitter.github.io/tree-sitter/creating-parsers#external-scanners) > Many languages have some tokens whose structure is impossible or inconvenient to describe with a regular expression. > Tree-sitter allows you to handle these kinds of tokens using _external scanners_ . An external scanner is a set of C functions that you, the grammar author, can write by hand in order to add custom logic for recognizing certain tokens. So the only required thing is specifying the language gramma in JavaScript DSL which is in Ruby case is less than 1000 lines. ---
**Author:** ![vlazar](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/vlazar/32/31_2.png) [@vlazar](https://forum.crystal-lang.org/u/vlazar)\ **Post date:** [July 30, 2021, 4:57am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/10 "2021-07-30T04:57:03Z")
As being already mentioned here parser is actually generated from grammar described in three-sitter JS DSL. Another cool thing with tree-sitter is there are already WASM bindings and once you’ve created parser with tree-sitter you can use it on the web [tree-sitter/README.md at master · tree-sitter/tree-sitter · GitHub](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/README.md) ---
**Author:** ![vlazar](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/vlazar/32/31_2.png) [@vlazar](https://forum.crystal-lang.org/u/vlazar)\ **Post date:** [July 30, 2021, 5:11am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/11 "2021-07-30T05:11:49Z")
Some meaningful(les) stats for parsers automatically generated by tree-sitter: - 52k lines for Go [tree-sitter-go/parser.c at master · tree-sitter/tree-sitter-go · GitHub](https://github.com/tree-sitter/tree-sitter-go/blob/master/src/parser.c) (and not hand written scanner in C) - 53k lines for Python [tree-sitter-python/parser.c at master · tree-sitter/tree-sitter-python · GitHub](https://github.com/tree-sitter/tree-sitter-python/blob/master/src/parser.c) (and 395 lines for hand written scanner in C) - 238k lines for Ruby [tree-sitter-ruby/parser.c at master · tree-sitter/tree-sitter-ruby · GitHub](https://github.com/tree-sitter/tree-sitter-ruby/blob/master/src/parser.c) (and 932 lines for hand written scanner in C) Fo those of you written a parser or 2 in your life ;) is it a lot lines of code than a hand written parser in C would have? Source grammars (terse nested JSON with not a lot code per line) - 6529 lines for Go [tree-sitter-go/grammar.json at master · tree-sitter/tree-sitter-go · GitHub](https://github.com/tree-sitter/tree-sitter-go/blob/master/src/grammar.json) - 4953 lines for Python [tree-sitter-python/grammar.json at master · tree-sitter/tree-sitter-python · GitHub](https://github.com/tree-sitter/tree-sitter-python/blob/master/src/grammar.json) - 6128 lines for Ruby [tree-sitter-ruby/grammar.json at master · tree-sitter/tree-sitter-ruby · GitHub](https://github.com/tree-sitter/tree-sitter-ruby/blob/master/src/grammar.json) ---
**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\ **Post date:** [July 30, 2021, 8:45am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/12 "2021-07-30T08:45:54Z")
> [@asterite](#): > > Though it’s strange… where does the error live? It looks like there’s a `binary` AST node, but it has an `ERROR` right after `left` and before `right` . I can recommend watching the talk. [Last 10 minutes](https://youtu.be/Jes3bD6P0To?t=1597) are the most interesting parts about error handling. ---
**Author:** ![wontruefree](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/wontruefree/32/1467_2.png) [@wontruefree](https://forum.crystal-lang.org/u/wontruefree)\ **Post date:** [August 18, 2024, 1:02am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/13 "2024-08-18T01:02:01Z")
I am refreshing my vim setup. Has there been any progress on a tree sitter plugin? ---
**Author:** ![nobodywasishere](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/nobodywasishere/32/1983_2.png) [@nobodywasishere](https://forum.crystal-lang.org/u/nobodywasishere)\ **Post date:** [January 4, 2025, 6:43am UTC](https://forum.crystal-lang.org/t/tree-sitter-crystal/3565/14 "2025-01-04T06:43:47Z")
Yes, see [GitHub - crystal-lang-tools/tree-sitter-crystal: tree sitter parser for crystal lang, fork of keidax/tree-sitter-crystal](https://github.com/crystal-lang-tools/tree-sitter-crystal). We’re sitting at ~75-87% of the stdlib parsing without errors, and making rapid progress. Upstreaming to editors (zed, neovim, helix, emacs, etc) should happen soon after we reach \>95-99%.