Thursday, 28 October 2010

TDD: The Starwars Answer vs The MacGyver Principle

"But what if you write a bug - like a typo or something - into the test? I mean, your tests aren't going to be perfect, are they?"

This is a genuine question someone - understandably - asked me earlier this week when I was introducing them to the idea of TDD.

I have two answers to this question:

1) The math answer:

If P(Error) is the probability of any line of code you write containing an error, and you write t lines of test code and tn lines of production code, where n is greater than 1, then bugs in both tests and production code can only slip through when they coincide, and you will fix tn bugs in production code for every t bugs in the tests.

And as P(A + B) = P(A) * P(B) then you vastly reduce the number of undetected bugs caused by genuine code errors - eg accidental assignment, wrong operations, wrong variable names etc.

If you don't speak Math, the outcome is many, many fewer small bugs.

2) The Starwars Answer:

"These are not the bugs you are looking for."

No - really, typos and wrong negatives are cool to find, and do cause many wasted hours debugging, but this isn't why I do TDD.

TDD is most powerful when it really hurts. The pain comes because you realise that you have written, or planned to write, some code that is hard to test. TDD pokes you right in the coupling. It stabs you with your own statics and shoves your crappy assumptions right...

... you get the idea?

When you write a test for a class, or a unit of functionality, you are trying to do that outside of the application. So all those hooks and shortcuts and bits of spaghetti that seem so handy inside your app swiftly start to look like what they are: problems.

But there's something TDD brings to your attention that's even harder to swallow...

TDD goes against the MacGyver principle

TDD exposes to you the numerous pathways through your code. If you're testing something as simple as a form with a set of radio buttons, you have to test every possible option for that radio button selection. You imagined you were going to write half a dozen tests for this class and suddenly you've got 15. And you can still think of more special cases that aren't covered.

This is a head-fuck because we like to pretend, to ourselves, that what we're trying to do is much, much easier and less complex than it actually is. I suspect there are very few genuine pessimists* in programming. We are self-selected MacGyver types with a tendency to see the solutions in any situation.

This process relies on the confidence trick of pretending to yourself that what you're undertaking is pretty straightforward, really. We have to turn a blind eye to the factorial expansion of pathways through our code presented by each new 'tiny' feature, because the alternative is lying awake at night wondering whether we've really covered every important combination of user actions, and knowing that we're never going to hit the deadline.*

Many flash programmers are just as badass as MacGyver. They bravely code ahead, pushing to the side the concept of failure - they know they can cross between these two structures using only a coathanger and some magic strings (static) because there is no room for any other possibility.

And hell, it works out a lot of the time. We all built some awesome stuff in as1 when we didn't even have compile-time-checking or type safety. But if you're still MacGuyver coding today, I have one piece of advice: Don't look down.

* There are plenty of us who are cynical, but I think that's distinct from pessimism. We probably believe there are solutions to most problems, but that people are too damn stupid / greedy to allow them to be implemented.

** In reality, nobody ever hits the deadline with MacGuyver coding, but they at least agree to the deadline, which most of the time is all the boss is looking for. With TDD, nobody agrees to the deadline the boss was hoping for. As always, TDD just shifts that pain forward in the timeline.


Vladimir Tsvetkov said...

I would definitely use the math explanation when asked the same question.

Here's one thought I read yesterday in Grady Booch's "Object-Oriented Analysis and Design with Applications" which left me in total despair about the character of our work as software engineers:

"The distinguishing characteristic of industrial-strength software is that it is intensely difficult, if not impossible, for the individual developer to comprehend all the subtleties of its design. Stated in blunt terms, the complexity of such systems exceeds the human intellectual capacity. Alas, this complexity we speak of seems to be an essential property of all large software systems. By essential we mean that we may master this complexity, but we can never make it go away."

That's the spirit!!!

Stray said...

Ha - that's a brilliant, brilliant quote! I think it's true that there are limits to what we can hold in mind at any one time.

I think one benefit of TDD is that it lets you set aside - once covered by tests - complexity that otherwise has to occupy human-ram.

We are now definitely working in a time where the constraining processor is the programmer's brain.

The funny thing about complexity is that the less able you are to comprehend it, and work with it, the less you even notice it. Aged 13, my kid and his friends wanted to know how long it might take, from scratch, to build one of the real-time multiplayer online games they love. They had variously estimated a long weekend to a whole summer holiday...