Monthly Archives: August 2009

Understanding test doubles

There is a bewildering array of types of ‘mock’ object available to a tester.  The canonical list of test doubles was probably coined by the venerable Martin Fowler in his article “Mocks Aren’t Stubs” and, to me, this list is fairly complete and makes sense.  The reason it makes sense is that I’ve manually written classes that perform these roles.

  • I needed to fill out a parameter list with non-null objects, so I created a dumb class with absolutely no implementation.
  • I wanted to listen in on additions to a list, so I wrote a class that stored the objects, acting as a spy.
  • I wanted to provide canned results to test another part of my system, so I wrote a stub.
  • I needed to ensure a method was called, so I wrote a Mock.
  • I needed a coherent, fast implementation of a class, so I wrote a fake implementation.

My problem with testing terminology is that it’s harmful to newcomers, especially when the semantics affect the result of the test!  Not only is there a high barrier to entry when it comes to writing accurate, robust and maintainable tests, but the terminology is another unwelcome complication.

For this reason, I personally advocate keeping new testers away from mocking frameworks until they’ve become comfortable with state-based testing and hand-rolled a variety of their own test doubles.

Information Overload

Even when no additional frameworks are involved (i.e. when using vanilla xUnit), writing good unit tests involves a steep learning curve.  It’s easy to take a wrong turn and the quality of the tests written will improve only with experience/guidance.  I was not surprised when I read Roy Osherove’s blog and discovered that the majority of organisations’ attempts to embrace unit testing resulted in failure.

Mocking frameworks like Rhino Mocks are absolutely excellent tools, but it’s yet another thing to learn.  Suddenly the type of object (mock, strict mock, stub etc.) affects the result of the test.  It took me a week to get my head around it, so it doesn’t surprise me when I see newcomers totally abusing these frameworks.  Not only does this create a maintenance nightmare, but it sours their first taste of testing.

The problem is doubly hard to tackle, as Mocking frameworks allow you to write the same tests with fewer lines of code.  Developers who are new to testing see this and immediately try to use the mocking framework.  After all, only a fool would eschew such benefits, right?  Well, some people just ‘get it’ from the start.  I’m not one of these people and, in my experience, nor are most others.  Starting with interaction based testing and mocking frameworks is akin to throwing someone out of the back of a van that’s moving at 70mph and expecting them to start running when they hit the tarmac.  Chances are they’re going to land on their face.

The unfortunate result is that some folk tie themselves in knots.  They don’t understand the responsibilities of each type of testing object and, as a result, create extremely brittle or utterly pointless tests. If you have no idea of what you’re trying to achieve with a test, there is no point in writing it.  I once saw a question on StackOverflow featuring a confused fellow asking why his test did not work.  The poster was using Rhino Mocks and created a Mock of the class under test.  I.e. it was not a collaborator, it was the class under test and he was mocking it!  I suspect he was simply overwhelmed by trying to learn multiple new things.

Other things I’ve witnessed include developers writing obscure lambda expressions using c#, then chaining together RhinoMocks methods to perform something which somehow works.  When I pointed out that I could barely infer its purpose by reading the code and that writing a hand rolled stub would probably be a better idea, I was met with “yes, you’re probably right but I want to use Rhino Mocks”.

Warning bells should also start ringing when you return a mock and assert that its method was called by another mock which returns a mock which… errrr!  It’s much easier to make a dog’s dinner of interaction-based testing; a good grounding in state-based testing is essential.

One step at a time

  1. Learn to sit up before you crawl.  Write simple xUnit tests that involve state-based testing.  It doesn’t have to be great, isolated code.  Even writing tests that involve scores of classes is a good way to start.  Finer granularity is something that comes with experience.
  2. Crawl before you walk.  Start to experiment and find better ways of testing pieces of functionality.  Ask yourself whether the test is useful, maintainable etc.  Will other parts of the system break it if they change?  Can you make the components and tests themselves finer grained?  This stage should be about developing your sense of what constitutes a good test.
  3. Walk before you run.  Begin to experiment with different types of test doubles, but hand roll them.  Yes, it’s painful at times, but it will give you a better understanding of roles in tests and the different types of test doubles, even if you don’t have names for them yet.  Furthermore, constantly having to update your hand rolled stubs when disparate parts of your class changes will also give you an appreciation for the interface segregation principle.
  4. Finally, install a mocking framework and start sprinting.

    If you do sprint head-first into a wall, you will be better equipped to understand where you went wrong, as you understand the fundamentals.  You will also have a better grasp of the terminology, as it will be grounded in real, tangible code you’ve written.

    Getting up and running with Fluent NHibernate

    I’ve been meaning to try out NHibernate for a good ol’ while.  It’s a long-established and respected O/R M library and one of the authors (Ayende) writes a blog that I’ve read for a long time.

    Anyway, NHibernate is great, but its object => db mappings are a bit of a pain.  They are based on xml which is verbose, fiddly to write and the separation makes refactoring and testing mappings somewhat hard.  There are other ways to create mappings in code, such as via attributes, but this approach pollutes your business objects with DB specific code and still doesn’t help with the testing issue.  This and the lack of a LINQ to NHibernate are the only two main gripes I’ve heard about NHibernate.  The latter problem is getting solved for the next release.

    The weakly typed mappings solution is already most of the way there.  Step forward Fluent NHibernate.  Fluent NHbernate alleviates these problems by providing both convention based auto mappings and mappings created via strongly-typed code.  Lots of blogs and articles cover Fluent NHibernate; the point of this post is to point out a few gotchas that may occur when getting up and running.

    Firstly, the FluentNhibernate example project does not have all of the required assemblies when you try to run it.  If you check the InnerException message, it’s clear which assemblies are missing.  From memory, it’s one of the byte code .dlls.  Either set up an assembly reference or create a postbuild step to copy it.

    Moving on:

    When writing my own Noddy sample application, I followed This Tutorial and, while it is good, it misses out a few things:

    Gotcha #1:

    If you use SQLite to run your application and/or test your mappings, the version of SQLite provided with Fluent NHibernate is an x86 assembly.  If you have a 64 bit OS and fail to build your project in x86 mode, you’ll get various obscure error messages (instead of a BadImageFormatException or whatever .NET usually throws).  The solution to this particular problem is to set the project(s) to build in x86 mode.

    Gotcha #2:

    The following line will also cause an exception (or at least it did on my PC — running 64 bit Windows 7):

    Id(c => c.Id).GeneratedBy().HiLo(“customer”);

    Again, it’s a very vague exception.  The article has someone seeking help for the same problem, but no solution.  I asked on StackOverflow and the solution is to either remove the trailing .GeneratedBy…. fluent method calls, or to replace it with something like HiLo(“1000″).

    There may be implications for making such a change, but when you just want to get a Noddy application up and running so you can do a bit of fiddling about, it’ll do the job. :)