Introducing A Cross-Platform Debugger For Go

We use Go for a lot of our server development here at Mailgun, and it’s great. Coming from Python, though, there is one thing I really missed:

I could insert this little line whenever I was confused about what was happening in the code and see exactly what was going on. But in Go? Not so much. When I started this project in January, gdb failed on every program I tried it on. delve didn’t work on OS X, and print-statement-debugging was too slow and limited. What’s a developer to do?

Make my own debugger, of course.

godebug

All that stands in the way [of a good Go debugger] is the writing of a lot of non-portable low-level code talking to buggy undocumented interfaces.

– Rob Pike

godebug is a different kind of debugger. Traditional debuggers for compiled languages use low-level system calls and read binary files for debugging symbols. They’re hard to get right and they’re hard to port.

godebug takes a different approach: take the source code of a target program, insert debugging code between every line, then compile and run that instead. The result is a fully-functional debugger that is extremely portable. In fact, thanks to gopherjs, you can run it right here in your browser!

You can edit the program and relaunch it with the “DEBUG IT!” button as many times as you like.

How did that work?

Here’s a quick diagram of the above demo:

The original code gets transformed twice. First, godebug inserted debugging instrumentation. Then gopherjs compiled the result to javascript.

Let’s take a look at the instrumentation step. (To learn more about the gopherjs part (which is awesome) checkout its website.) Here are some of the calls that godebug inserts:

• godebug.EnterFunc lets the godebug runtime library know that we are entering a function. Since “next” doesn’t stop inside function calls, the runtime library takes note of these calls so it knows when to skip over lines.

• godebug.ExitFunc lets the runtime library know we are leaving a function. Omitted in main.

• godebug.Line causes the program to pause and wait for input if and only if a user command or a breakpoint told it to. When it pauses, it prompts the user for input and responds to any commands.

• godebug.Declare records the mapping of variable names to their values. This mapping is used by the print command.

This is an abridged overview. There are other functions that godebug inserts and many details of the above functions have been omitted. But these are the basic pieces of how godebug works.

Using godebug

All of the above (minus the server & javascript stuff) is packaged into the godebug command line tool. Here’s how to use it:

Step 1. Install it

$ go get -u github.com/mailgun/godebug

Step 2. Set breakpoints

Add this marker anywhere you want a breakpoint:

_ = "breakpoint"

This statement becomes a breakpoint when running under godebug and is a no-op otherwise.

Since breakpoints are part of the source code, you can wrap your own logic around them. Let’s say you are running a table driven test with dozens of cases, and one of the test cases fails: the one that tested the input "weird string". You can add this breakpoint to your test:

for _, tt := range myTestCases { if tt.in == "weird string" { _ = "breakpoint" } ... }

godebug test will pause the program at the marker statement, which is conveniently positioned just before the failing test case runs.

Step 3. Run your program

Use the godebug run command:

godebug run <gofiles...>

Or, for tests, use the godebug test command:

godebug test

By default, godebug will only add debugging instrumentation to package main (for godebug run) or the package under test (for godebug test). It will not instrument any imported packages. This is to decrease the overhead of the debugger — any packages you are not interested in will run as normal, at full speed.

This means that by default you can not step into function calls from imported packages. But sometimes you will need to do that! To debug other packages, pass the -instrument flag to godebug run or godebug test:

godebug test -instrument=pkgA,pkgB,pkgC pkg/under/test

The above command will instrument (and thus allow you to step into) pkgA, pkgB, pkgC, and pkg/under/test. It will then build and run the tests for pkg/under/test. Similar semantics apply for godebug run.

Try it out!

Next time you want to understand what is happening in a Go program, try godebug. Keep in mind that it is still a new tool that needs some polish. In particular, some known limitations are:

• performance overhead

• may cause read conflicts if your program reads from stdin

• can’t attach to a running process

• must know the packages you want to debug before starting the session

That said, I’m excited about this tool and hope it can provide a lot of value to the Go community. Try it out and let me know what you think! If you have any problems I would be happy to hear about them. File an issue at https://github.com/mailgun/godebug/issues or send an email to: dev@godebug.rocks

Hope you enjoy it!

https://github.com/mailgun/godebug

Want to learn more about godebug? Here is a talk I gave at GoSF, including more depth on code generation and concurrency management: