Mark’s Testblog

.. I’ve swapped jobs at Realtime Worlds; I’m now a plain ol’ software engineer.  As a result, there’ll be no more test engineering for me.

Whilst it’s true that I am changing jobs, what I learned as a test engineer has irrevocably changed the way I write software for the better.  I learned about the value of automation and wrote tools to automate processes, but the most satisfying thing I did was learn about how to design for testability.  Not only do these principles aid us in automated testing, but I firmly believe that following these principles results in better code quality.  These two separate things have much in common and seem to naturally converge.

Any time I had to pick through some code, the more thought that went into the testability of the code, the easier it was to work with and reason about the code.  As such, doing a two year stint as a test engineer was possibly one of the best entry level routes I could’ve hoped for.

I’m test infected and there is no going back on that.

One of the things that really slashes the return on investment in testing is unreadable code.  “This is pretty obvious”, you say.  “What’s the point in a blog post about something so obvious?”  What’s not obvious is that the very people writing these tests are unaware of it.  Maybe you do it as well.  Given that this blog post is about things we don’t know we do, I think it’s a fair bet that I’ve also recently written test code that was convoluted without realising it, too.

It’s mostly down to testing in a vacuum.  The tests are often functionally fine.  However, as with nearly all code, maintenance is easily overlooked.  On the day that the test was written, it made sense to the author.  They understood the logic they wanted to test and how to implement it.  Code written, job done.

In the court of the test engineer, readability is king

In my opinion, readability in tests is the number one thing.  If your fellow programmers and your future self cannot decipher what a test is proving, the test becomes worthless.  If a monstrously bearded mathematics genius solved solved P versus NP but failed to write an understandable proof, it would all be for nothing.

It’s the same for testing.  You’ll know the shit has hit the fan when you’re refactoring something and break a load of tests.  “Balls.  I’ll fix the tests I guess”, you murmur, bleary eyed and idiot-faced.  However, when you examine the tests, you can’t figure out what they’re proving, why they’re proving it or how it’s actually proved!

You know in films when a bad guy is holding a hand grenade, then has a moment of realisation regarding an absent pin?  He slowly looks up, face furnished with a quizzical look, staring into oblivion.  That’s what a software engineer looks like when they encounter reams of broken tests that cannot be deciphered (and the person who wrote ‘em isn’t available to pick up the pieces).

Grab a friend and test your tests

To avoid these kinds of bomb-scares, do yourself a favour and have somebody else trawl through your test code without giving them any help.  If they cannot easily follow the test code’s intent, then the test needs re-written.  Everybody’s code can be improved through peer review.  Peer review is the litmus test — if somebody else cannot understand it, it’s useless.  As such, tests should be part of code reviews and buddy processes.

Think about how much time we spend reading, maintaining, refactoring and extending old code compared to writing new code, and then think about how self-defeating it is to write unreadable, un-reviewed test code.  Spending a little more time on reviewing new test code pays off in the long run.

Prepare for some flimsy and strange analogies.

I’ve been reading a few stackoverflow questions dealing with whether you should test the guts of a system as well as the public API.  Most of the people who advocated never testing anything but the main class APIs seemed to talk as if these APIs were extremely coarse-grained and any change to the internals would result in major breaking changes to the tests.  This strikes me as somewhat strange, as it’s often not the case in my (admittedly limited) experience.

On the other hand, many of the developers who advocated testing the implementation details also advocated test-driven development (TDD); that’s when the penny dropped.  To me, this is a good illustration of why designing for testability can make change less painful.  Sometimes I cringe when I hear the phrase “agile” bandied about, but it rings true here.

Little, bitty pieces

Designing for testability in conjunction with TDD tends to produce loosely coupled classes that have very few responsibilities.  You trade more complicated wiring and interactions for unit isolation and simplicity.  In the majority of cases, I feel it is an attractive proposition.  Instead of one 3000 line class that does everything, I end up with 25 x 50 line classes and the odd bigger one here and there.

Complicated behaviour is usually achieved through composition (inversion of control using constructor injection) and delegation.  I consider most of my inner classes to be implementation details, because the user doesn’t get to do anything with them.  They’re off sitting in a library somewhere; they’re not exposed to the user.  The user gets a hold of the top level class that tells the innards to do the real work (instead of the 3k line class that does everything).  I can take any unit in the system off the shelf and test it without a struggle.

So, who is going to suffer more breaking changes and hardship if they test the internals?  Is it the developer who writes the all-singing, all-dancing monolithic class, or the developer who writes numerous, small, simple classes that can easily be tested in isolation without any fuss?  Since you’re reading a testing blog, I think you know what I’m going to say, and it’s not going to be the twisted brain-wrong of a one-off man mental*.

Big is awkward

Large classes are harder to understand, maintain, refactor and test thoroughly.  Furthermore, it is my experience that the biggest classes tend to grow and grow.  The bigger it grows, the more ungainly it becomes.  Gangly limbs poke out every side; sharp edges are present in abundance.  It’ll probably stick the heid on you or punch you into paralysis.

Have you ever picked up a huge class and been tasked with adding new functionality and testing it while you go?  It’s painful.  By the time you’ve worked out which tightrope you’ve got to walk (and fallen off it 20 times due to the principle of most surprise), you’ve wasted a lot of time adding in the new functionality and even more time writing the tests.

Small is beautiful

Contrast that to a system where you just add a method or two in a simple class (or add a new one) and the maintenance headache is reduced to a dull ache.  If it’s easy to write, it’s easier to understand, test, maintain, refactor and — just as importantly — it’s even easier to throw away.  I don’t get attached to tiny classes or their respective unit tests.  They’re like tic-tacs; if I lose one or five, I shrug.  Big deal.  I get some new ones.  Open for delete.  Don’t cry you buffoon, it’s just a tic tac.

Misko Hevery recently posted something interesting on his testing breakdown and, while most of us won’t reach his level of testing efficiency, it’s an interesting read.  Misko states that the vast majority of his time is spent writing production code, not test code.  Yes, the ratio of lines of test and production code produced  is almost 1:1, but the time invested is wildly different.  Test code is usually verbose, but it’s easy to write out when your classes are small and you test in lockstep.

In summary, I believe testing the guts of your classes can be a worthwhile approach, but designing for testability is paramount when doing so.

*