calling a function inside visual D debugger: partially possible, want better support

[timotheecour]

short version:
how to call a (non extern-C) function from visualD's debugger? (
I can do it with extern C)
how to call a function with arguments that are stack variables (I
can do it with actual numeric values, eg actual pointer addresses)

long version:

when debugging a session inside visual D, it can be very useful
to call functions directly, eg:

* to print variables that visual D doesn't display properly (such
as dynamic arrays, where visualD just prints the length and
pointer)

* for interactive purposes (eg trying a function with different
parameters)

Currently, visualD's support for this is limited, but I was able
to do certain useful things (I'm not sure if many people realized
one can do this in visuald, but this can be useful):

suppose we compile a D program into Main.exe:
we can call extern(C) functions as follows, by adding a watch
variable in visual studio debugger and entering in the name field:
Main.exe!main_test at test1(2)
=> will show value=25, type=int, and will print in the command
prompt "res=25"

now get address of z1 by looking into "Locals":
it shows:
- z1 {length=4 ptr=0x01f20f40 {1} } int[]
then display the dynamic array z1:
Main.exe!main_test at writelnPtr(3,0x01f20f40)

This is nice but not very convenient. Is there a way (if not, can
that be addressed in the near future) to support the following?

* Main.exe!main_test at writeln2(z1)
=>currently returns: identifier "z1" is undefined

* Main.exe!main_test at writeln2([1,2,3])
=>currently returns: expected a type specifier

* Main.exe!main_test at writeln3()
=>currently returns: identifier "main_test at writeln3" is undefined

I also tried with the mangled name of writeln3 but that didn't
work with visuald D. However, it DOES work on OSX, after
compiling with debug symbols and then running under lldb:

b _Dmain
r
(lldb) expr (int) _D5tests16test_scratch_tim8testfun5FiZi (4)
=> doesn't work
(lldb) expr (int) D5tests16test_scratch_tim8testfun5FiZi (4)
=> successfully runs the underlying function once we remove the
leading underscore

----
module main_test;
void main(){
int[]z1=[1,2,3];
//breakpoint here
writeln(z1);
}
extern(C)auto test1(int x){
auto y=x*x;
writeln("res=",y);
return y;
}
extern(C)void writelnPtr(size_t n,void*ptr){
writeln((cast(int*)ptr)[0..n]);
}
extern(C)void writeln2(int[]z){
writeln(z);
}
void writeln3(){
writeln("test");
}
----

[Rainer Schuetze]

how to call a (non extern-C) function from visualD's debugger? ( I can
do it with extern C)
how to call a function with arguments that are stack variables (I can do
it with actual numeric values, eg actual pointer addresses)
when debugging a session inside visual D, it can be very useful to call
* to print variables that visual D doesn't display properly (such as
dynamic arrays, where visualD just prints the length and pointer)
* for interactive purposes (eg trying a function with different parameters)
Currently, visualD's support for this is limited, but I was able to do
certain useful things (I'm not sure if many people realized one can do

I know it's possible and I use it in C++ from time to time, but never
tried it in D.

we can call extern(C) functions as follows, by adding a watch variable
Main.exe!main_test at test1(2)
=> will show value=25, type=int, and will print in the command prompt
"res=25"
- z1 {length=4 ptr=0x01f20f40 {1} } int[]
Main.exe!main_test at writelnPtr(3,0x01f20f40)
This is nice but not very convenient. Is there a way (if not, can that
be addressed in the near future) to support the following?

VS2012 allows "main_test at writelnPtr(z1.length,z1.ptr)"

Actually, the changes to autoexp.dat that are done by the installer are
supposed to display dynamic and associative arrays, but it seems they
are broken in VS2012 and maybe also 2010.

* Main.exe!main_test at writeln2(z1)
=>currently returns: identifier "z1" is undefined

VS2012 says: "Passing class, struct or union types by value to a
function evaluation is not supported". It works if you pass the argument
by reference.

* Main.exe!main_test at writeln2([1,2,3])
=>currently returns: expected a type specifier

The VS debugger expects C/C++ syntax, so it cannot do this.

* Main.exe!main_test at writeln3()
=>currently returns: identifier "main_test at writeln3" is undefined

If you disable name demangling by cv2pdb, the symbol
_D9main_test8writeln3FZv resolves to the function writeln3, but
unfortunately its type is reported as "void*". I guess the VS debugger
does not allow the calling convention used by D in function evaluation.

If you use mago as the debug engine, you will get better D expression
support and output, but it is not capable of running functions through
watch expressions. Unfortunately development of mago seems to have
stopped/paused. Nevertheless, I think a dedicated debug engine is the
way to go for the best user experience.