The Goldfish Chronicles

While hacking on a small bit of code today, I did something I often do when hitting a bug and felt embarrassed.

The technique is what I’m going to call Blackbox Debugging; I’m 100% sure it’s not an original idea, but it’s not one that I ever see talked about much so I wanted to dump some thoughts on it. In today’s environment where the source of an API is increasingly available, it’s too easy sometimes to go get lost into how the code actually works. Sure it’s the best way to know that code, but sometimes it’s not possible, and sometimes the work to do that isn’t worth the effort.

On these occasions, I like to do something that looks remarkably like randomly changing things and hoping things work. That’s why it’s embarrassing, it feels bad, but while feeling a bit bad about doing it today I pondered it a bit in the spirit of Beautiful Code and realized that it was a damn good trick.

A lot of text for something that is probably patently obvious to y’all. Blackbox Debugging is the act of perturbing the input to the API to learn more about what’s happening. Let’s say you’re calling a function square(int), and when you pass in 5 you get back 9. Bwah! WTF! etc.

Try 6. Get back 11. You’ve just learnt something crucial - the function is input dependent. It’s not just “return 9;” AND it’s still broken.  Try 7. Get back 13. Increase input, and output seems to increase. Whatever. It’s building data. Now that I think about it more, this is the zeroth case for the Tracer Bullet pattern. The important bit being that you don’t start by diving into the code and inserting trace statements (or diving into the debugger), instead you build up some data so that when you do dive in to build your trace/breakpoints, you have a much clearer picture of what you’re looking for.

The most important thing of course is that if the code then works - don’t let it lie. A success at this point is the worst thing to see and the code is definitely wonky. Badger it to death until you grok wtf has been going on.

Anyway, thought I’d share the thought.

This entry was posted on Friday, February 8th, 2008 at 11:26 pm and is filed under Opinion. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.