Debugger support

@asterite Ary what is the best way of reaching you? I am not able to send you direct message.

I have some questions re compiler internals that still puzzles me and I wanted to discuss it in private. Can you send me your contact information via direct message here at forum?

You can write to me here. The problem is, I donā€™t have time to do anything related to Crystal other than browsing the forum, checking GitHub and replying small bits from time to time. Sorry :disappointed:

@asterite I do understand. No issues at all. I need your help with understanding how block variables are defined. I have some interesting effect when my block arguments are shown not in the block but in the parent scope of def each (in the scope of yield). I localized that arguments code in codegen.cr around line 1480 ( I added declare_variable call after line 1480:

unless block_var.debug_var
     declare_variable arg.name, block_var.type, block_var.pointer, block_var.location
end

block_var.debug_var is my flag that tracks if variable has debug variable value already created or not.
If I use exp_type, exp_value instead of block_var.type, block_var.pointer then it shows in correct place but no values :frowning:

Also what closured variable means in crystal codegen context? Is it variable that captured from outside scope? Does alloca_non_closured_vars should allocate block variables?

I factored llvm module a little by exposing enumerables for basic blocks and instructions as well as accessing for instruction metadata so now it can be much easier to work directly on per alloca variable info data as per this example:

   def body_to_ll_string
     String.build do |str|
       str << "def " << self.name << "(#{self.params.join(", ")}):\n"
       self.basic_blocks.each do |block|
         str << "Block " << block.name << ":\n"
         block.instructions.each do |inst|
           str << "\t" << inst << " -> metadata: " << (inst.instruction_metadata || "N/A") << "\n"
         end
         str << "\n"
       end
     end
   end

which in turn provides this debug snippet:

</Users/sergey/Projects/crystal/crystal/src/compiler/crystal/codegen/codegen.cr:368> def *UInt64::new<UInt64>:UInt64(i64 %value):
 Block alloca:
           call void @llvm.dbg.declare(metadata i64 %value, metadata !19768, metadata !DIExpression()), !dbg !19769 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
           br label %entry -> metadata: N/A
 
 Block entry:
           %0 = icmp ult i64 %value, 0, !dbg !375 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
           %1 = icmp ugt i64 %value, -1, !dbg !375 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
           %2 = or i1 %0, %1, !dbg !375 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
           %3 = call i1 @llvm.expect.i1(i1 %2, i1 false), !dbg !375 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
           br i1 %3, label %overflow, label %normal, !dbg !375 -> metadata: !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1019, column: 3, scope: <0x7f9912cbfbb0>)
 
 Block overflow:
           call void @__crystal_raise_overflow(), !dbg !376 -> metadata: !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>)
           unreachable, !dbg !376 -> metadata: !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>)
 
 Block normal:
           ret i64 %value, !dbg !376 -> metadata: !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>) = !DILocation(line: 1020, column: 11, scope: <0x7f9912cbfbb0>)
 
 </Users/sergey/Projects/crystal/crystal/src/compiler/crystal/codegen/codegen.cr:368> def *GC::realloc<Pointer(Void), UInt64>:Pointer(Void)(i8* %ptr, i64 %size):
 Block alloca:
           call void @llvm.dbg.declare(metadata i8* %ptr, metadata !19773, metadata !DIExpression()), !dbg !19775 -> metadata: !DILocation(line: 119, column: 3, scope: <0x7f9912cc1480>) = !DILocation(line:  119, column: 3, scope: <0x7f9912cc1480>)
           call void @llvm.dbg.declare(metadata i64 %size, metadata !19774, metadata !DIExpression()), !dbg !19775 -> metadata: !DILocation(line: 119, column: 3, scope: <0x7f9912cc1480>) = !DILocation(line: 119, column: 3, scope: <0x7f9912cc1480>)
           br label %entry -> metadata: N/A
 
 Block entry:
           %0 = call i8* @GC_realloc(i8* %ptr, i64 %size), !dbg !379 -> metadata: !DILocation(line: 120, column: 5, scope: <0x7f9912cc1480>) = !DILocation(line: 120, column: 5, scope: <0x7f9912cc1480>)
           ret i8* %0, !dbg !379 -> metadata: !DILocation(line: 120, column: 5, scope: <0x7f9912cc1480>) = !DILocation(line: 120, column: 5, scope: <0x7f9912cc1480>)

It helps me to debug debug lines mismatch.

Are you okay if my changes can go into LLVM standard lib package?

I can help if I have time!

But could you post a single question in each post? I read it all but I donā€™t know what questions you have, or what problems there are. I can answer what are closured variables, but there are so many things you asked, I donā€™t have the time to answer them in a single post. So if you could write shorter posts with one question in each (and wait for my answer) that would be great.

Also, I didnā€™t write any of the debug code in Crystal so Iā€™m pretty much clueless when it comes to that.

A closured variable is something like this:

def foo
  a = 1
  ->{ a }
end

a is a local variable but itā€™s used in a proc, and that proc survives the lifetime of the method so a must live too. Therefore itā€™s allocated on the heap instead of just on the stack.

2 Likes

@asterite Cool!

So I see this is exactly what I expected.

In case of blocks and not closures variables inside of the block are defined in the current function context and they will be defined in context.vars or they will be defined in context.block.vars?

so call_with_block should handle all that cases or it is defined during visit(Yield) call?

My goal is properly acquire alloca pointers to variables, their names, scopes and exact locations for arguments and variables defined in the blocks.

PS: All other questions is partially answered by you or mostly informative.

All local variables should be in context.vars. context.block holds a reference to the current block, if any, and a block has a reference to the variables it uses (I think). But context.vars is info for codegen, context.block.vars if info from the semantic pass.

@asterite So context.vars will not have proper variable location information but context.block.vars will?
If so this is a very good information to know.
And if I understand correctly, I can walk up by context.block_context up to the top context that belongs to the function in case if there is a set of embedded blocks?

@asterite My another question was about pointer to the unions.
Do they point to type_id or to the second gep element (allocated array for the union data)?

@asterite what is the Block.scope for? it contains Type? class but how that info can be used?

1 Like

I donā€™t think I said that. I didnā€™t say anything regarding what has or doesnā€™t have location. Everything should have a good location. If not, itā€™s a bug (sometimes we forget to copy locations from vars).

And if I understand correctly, I can walk up by context.block_context up to the top context that belongs to the function in case if there is a set of embedded blocks?

Maybe, I donā€™t remember the code.

It dependsā€¦ a union will point to the type_id. The value of a union might point to the value. Do you have any code where you need this answered?

what is the Block.scope for?

I donā€™t remember right now, I think itā€™s an optional scope that is used if you do with exp yield. In that case the block will have the type of exp in scope.

@asterite Cool! I think I got it! Thanks a lot Ary!

Here we go!
After Ary explained me some of the intrinsics I was able to fix debug info for block variables.
Also bonus StaticArrays are shown as C arrays in debugger:

My next step is figure out how I can bind to @size valiable to show Arrays, Sets and Hashes in proper way as well.

7 Likes

@asterite I just committed last changes for debugger and I think it is ready for tests.

I need volunteers who will test it with LLDB and CodeLLDB plugin and also want some volunteers to test it with GDB.
Any ideas and suggestions are welcome!

@bcardiff I will look to your suggestions for debugging unit tests with GDB and LLDB (I have working LLDB only so I will concentrate on it). I also planning to do the testing on LLVM bytecode level after bytecode is generated from example.

@RX14 I would love your opinion on the PR and you can experiment with it and help me with some polishing,

1 Like

Hi @computermage do you have a a recipe for trying out the debugger?

I assume it should be clone your repo, compile crystal, compile a program load it on lldb or gdb?

@lribeiro Yes, clone my repo (https://github.com/skuznetsov/crystal) and switch to debug branch, then compile as usual.
The generated binary with debugger support will be stored in .bin/ folder you can use that path for compiling your programs with ā€“debug flag added.
Compiled programs should have all necessary debug information for LLDB and GDB to be able to debug your program.
Due to the LLDB bug you may see stack variables (like local primitives variables) to shown incorrect values.
If you testing LLDB in Visual Studio Code you may install CodeLLDB plugin that shows variables neatly (you can see it on my screenshots in this thread). For GDB I suspect you will have to use Native Debug plugin but I donā€™t really like it for LLDB.

3 Likes

Now, with debugger support, it is very easy to work on my other Crystal projects. I just refactored most of my Java class reader over the weekend and more to come.

6 Likes

@asterite Ary, what are your plans with making Crystal v1.0? When we will have proper Windows support or earlier?

Crystal is not my language :slight_smile:

Iā€™m a contributor, like many others. I donā€™t lead the direction of the language so I donā€™t know the answer of those questions.

1 Like

@asterite But you are initiated creation of the language so you are lead contributor ;)

In my opinion it is ready for version 1.00

Shall we vote for versioning then? :)

Maybe you can create a repo with the modified LLDB (and compile instructions for it) or somethingā€¦looks nice!