Debugger support

So I just submitted request:

Thank you for your comment.

Subject: Requesting DW_LANG_Crystal for Crystal Language
Name: Sergey Kuznetsov
Email: sergey@i*******.com
Section: 7.12 Page: 230-231
Type: Enhancement

I am implementing proper debug support and would like to request a DW_LANG_* code for Crystal Lang

Section 3.1.1, pg 62
Section 3.1.1, pg 62
The Crystal language is described at https://crystal-lang.org/

Propose to add a new language name to describe Crystal.

In section 3.1.1, Table 3.1. Language Names, add DW_LANG_Crystal, Crystal to the list of supported languages.

In section 7.12, Figure 7-17. Language Encodings, add
DW_LANG_Crystal, 0x0028, default lower bound = 0

PS: I see that Kotlin and Zig languages are applying for 0x0026 code so to be on a safe side we will pick 0x0028

4 Likes

Wonderful!

Hopefully a 0x0028 patch can go upstream to lldb to make it work like C. Or perhaps even more!

Initially I will send a patch to use Clang AST Context (before that I will push fix for improper derefrencer).
Then we can start implement proper Crystal Language Plugin for LLDB. I want to investigate GDB plugins and if it is possible to implement it with Python API (both LLDB and GDB use Python for scripting API, I used only LLDB for now as GDB does not work for me).

Here is official link to the issue (just shown up):
http://dwarfstd.org/ShowIssue.php?issue=200120.1

Interesting fact: 0x28 is somewhat reverse mirror for 0x8002 (if we remove zeroes in the middle), or simple flip of 8 from left to right., cyclic rotation to the left. ;)

This is super exciting I am am so done with using puts or raises to debug things.

1 Like

I know the feeling! I had to do a lot of them during fixing the debug support as well. I still do even I have debug support now as I have t be sure that I am showing correct information.
At this moment I am fixing the debug line info as quite often debugger jumps some unexpected lines so to be able to finish debugger support PR I have to handle it as well.

Feel free to send PRs to fix wrong debug information even if the the debugger story is not complete yet. Thanks for the ongoing effort!

1 Like

I am just crying of happiness! Now even with line debug mismatch and some quirks for some of the debug types it is still waaaaaay to easy to debug the code.

Here is my hunt for debug lines mismatch example:

8 Likes

@bcardiff I was able to figure out what was the issue with debug lines mismatch:
it was due how yield and blocks are inlined into the Def function. I had to modify fun_metadatas to incorporate stack of lexical blocks (with different files pointers) so it is able to find proper scope based on the context.fun and location.original_filename

Here is my version of it (in get_current_debug_scope):

    array = fun_metadatas[context.fun]?
    scope = nil
    if array
      array.each_with_index do |scope_pair, idx|
        if scope_pair[0] == location.filename
          scope = scope_pair[1]
          Crystal.debug_log do
            puts "get_current_debug_scope -> fun_metadatas[#{idx}]:\n\tloc: #{location}\n\tscope=#{dump_metadata(scope)}"
          end
          return scope
        end
      end
      file, dir = file_and_dir(location.filename)
      di_builder = di_builder()
      file_scope = di_builder.create_file(file, dir)
      scope = di_builder.create_lexical_block(fun_metadatas[context.fun][0][1], file_scope, location.line_number, location.column_number)
      array << Tuple.new(location.original_filename || "??", scope)
      Crystal.debug_log do
        puts "get_current_debug_scope -> fun_metadatas[#{array.size}]:\n\tloc: #{location}\n\tscope=#{dump_metadata(scope)}\n\tscope=#{dump_metadata(file_scope)}"
      end
    end
    scope

I am still testing it though.

This approach (that I mentioned above) definitely fixed issue with wrong debug lines. So I will test more and will create a proper PR for this specific case.

2 Likes

I guess it sounds reasonable.

But I would expect that the caller method will point to the yielding method source for the inlined instructions. Why the callee method needs to know about the lexical contexts?

There was a previous PR to add some testing for the debug information. If something like that can be included to ensure this feature will not break it would be awesome. It does not need to be fast or integrated in the compiler_spec. But it would need to be scriptable.

This is the way DWARF keeps information. One scope is connected to the file scope. if file is different than original one you have to create a new lexical scope that will point to new file and that scope will have a line numbers:

!659 = !DIFile(filename: “string.cr”, directory: “/Users/sergey/Projects/crystal/crystal/src”)
!1629 = !DIFile(filename: “char.cr”, directory: “/Users/sergey/Projects/crystal/crystal/src”)
1778 = distinct !DISubprogram(name: “dump”, linkageName: “dump”, scope: !1629, file: !1629, line: 502, type: !620, scopeLine: 502, spFlags: DISPFlagLocalToUnit | DISPFlagDefinition, unit: !0, retainedNod
es: !1779)
!1779 = !{!1780}
!1780 = !DILocalVariable(name: “self”, arg: 1, scope: !1778, file: !1629, line: 502, type: !1601)
!1781 = !DILocation(line: 502, column: 3, scope: !1778)
!1782 = !DILocation(line: 503, column: 5, scope: !1778)
!1783 = !DILocation(line: 536, column: 7, scope: !1778)
!1784 = !DILocation(line: 268, column: 5, scope: !1785)
!1785 = distinct !DILexicalBlock(scope: !1778, file: !659, line: 267, column: 18)
!1786 = !DILocation(line: 525, column: 10, scope: !1778)
!1787 = !DILocation(line: 526, column: 10, scope: !1778)
!1788 = !DILocation(line: 526, column: 13, scope: !1778)

I am reading some links where they recommend to use DILexicalBlockFile instead of DILexicalBlock so I will play with it.

Ok. I pushed my fixes to my branch so now it shows block variables as well and it properly dives into the methods with blocks (before it was jumping randomly within the same file due to the file scope issue):

So if someone feels adventurous then just follow instructions in this thread above how to fix LLDB (and recompile it) or try it in GDB (I didn’t tried yet as it does not work on my macOS) and you will be handsomely rewarded with debugging experience.

PS: There are still some glitches with debug info that needs to be ironed out but it is mainly useable.

3 Likes

@ComputerMage Regarding the yielding methods debug information, something I would expect to work is to treat them as inline method. At the end of the day, they are inlined during the codegen. With this approach the caller method will contain debug location for itself and the called one. Does this approach about using DW_AT_call_file and DW_AT_call_line make sense to you for this case?


Regarding the debug branch. Some minor feedback to match the code base:

  1. Crystal.debug_log could be rewritten in a similar way as debug_codegen_log method.
  2. All the new LibLLVMExt methods should have snake case name (I know they do in LLVM::DIBuilder)

In Rust they interpret a bunch of test case scripts (some of them have both lldb and gdb instructions).

The approach used by Rust has the advantage regarding the PR I mention before: It is easier to check manually some specific test case.

The effort put here is great and I really think we need some test cases to ensure we don’t go backwards. (I’m not asking for you to do it)


Many of the DW_LANG request delayed aprox month before been accepted. :crossed_fingers:
I wonder what needs to happen after that so people have a compatible lldb version.

I will be very happy to work out the rest of the incorrect debug info bugs, since it enables using kcov for coverage reports of the stdlib spec suite, and makes stuff like coz.cr work correctly

This is a good idea, I will reimplement it and I will move the code next to it. Or I can even just reuse it as they are kinda similar.

Will add function alias to it.

I am investigating how are LLVM is generating those DIEs. Looks like I have to setup is_inlined flag on the lexical scope to make them generated.

We already can use identifier. As long as they not clashing with others, this is the reason why I skipped 0x27 as it may be picked up by either Kotlin Native or other contender.
What we need to do is just patch to LLDB to be accepted and probably the same for GDB

Feel free to experiment with my branch! It will help me tremendously as you know memory layout of types much better than me at this moment ;)

So we need to send the patch to both after the id is accepted, right?

And then we need to expect the users to update to the latest lldb/gdb version for that patch to be available, and that could take a while I guess. I am not sure if the ids can be updated somehow in current installations or we need to wait a release cycle. #DumbQuestionsSorry

No, we don’t have to wait when lang id will be accepted. We claimed it first and most likely it will be awarded. If you can see above response to me from dwarfstd.org maintainer you will see that it is not necessary. Kotling guys are waiting for more than a year (almost two years) to accept it. It will be accepted when DWARF v6 will be published. So it is okay to use it as is as we publicly claimed that lang id.

I will need to send my other patch (for bug found by me) to LLDB guys as well to see immediate (not referenced) values. I didn’t tested GDB for that yet but I will check it on my Linux instance.

1 Like