Category Archives: software

Python data-driven testing, ddt and @unpack

Several eons ago, I wrote a blog post about using the python ddt package. My only criticism was that you had to manually pack/unpack the test case arguments.

Happily, the developers of ddt were nice enough to add an @unpack decorator! You can now provide lists/tuples/dictionaries and, via the magic of @unpack, the list will be split into method arguments.

Documentation is here: http://ddt.readthedocs.org/en/latest/example.html

Empathetic Code

I always liked the phrase, “if you’re incapable of empathy, then you shouldn’t be designing APIs”.

I’ve worked with some terrible libraries in the past and the worst ones all had a common design flaw: the author didn’t put themselves in the user’s place. Worse still, sometimes the author just didn’t give one. When someone using the code pointed out that something was hard to understand or didn’t make sense, they were met with a snarl and shouted down; “it makes sense; you’re just using it wrong”.

On the other hand, I’ve occasionally picked up libraries/frameworks that make an extra effort to guide you along the correct path. The author of the code understood where users were likely to stumble.

image

It doesn’t matter whether it’s a carefully designed API you’re crafting for external use or internal code that will only be viewed and modified by engineers inside the company, you can always do something extra, especially if it’s something that’s already tripped you up.

If you encounter a gotcha, why not write a bit of code to clearly document the case, particularly if it is time-consuming to debug in the first place?

Read more »

A R#/Visual Studio style keymap for PyCharm

PyCharm is a lovely little IDE for Python by the splendid JetBrains, but the built-in Visual Studio keymap is not much like the default VS/R# settings I’m accustomed to. I’ve rustled up a quick version that restores all of the easy to remember refactoring key combinations (Ctrl-Shift-R, Ctrl-R, Action Key etc.) along with some of the common navigation ones (Ctrl, comma for recent files and so forth).

There’s a few things missing and I’ve unbound some of the PyCharm functionality as it clashed with the R# style keymap, so your mileage may vary. Give it a try, though!

You can download it here: http://defragdev.com/files/RStyle.xml

Video recording with MSI Afterburner

I’ve used this application for half a year now, but had no idea that it has the ability to capture video. 

It’s slightly more configurable than FRAPS, the file sizes are smaller and, crucially, it doesn’t seem to degrade in performance when recording for multiple minutes.  My registered copy of FRAPS causes massive system lag after a few minutes of recording (as in, I’ll drop from 60 fps down to 3 fps, and it’ll just constantly ping-pong between 3 and 60 fps until I stop the recording).

MSI Afterburner works perfectly for me.  Give it a try

Functional-style programming – Easy debugging!

Ever since LINQ arrived in C# land, a lot of people have been using a functional style of programming without even perhaps being conscious of it. 

I always chuckle when I read the words “functional style”, because I picture grim marches across a barren landscape, brutalist architecture and people with beards. 

One particular facet of programming in a functional style is where, rather than mutating an existing piece of data, input data is transformed / projected into something new via a pure function.  Since this style of programming limits the number of side-effects, it makes debugging a comparatively wonderful experience. 

Read more »

The wonders of Debugger.Launch()

Ever worked on a project that involved spawning new .NET processes? (as in, one [arbitrary] program launches another .NET executable)  I’ve had to do this on quite a few occasions over the years and the one thing that always saves my bacon when it comes to understanding and fixing bugs is Debugger.Launch().

A common scenario is as follows:

  • Program / Script A is launched
  • Program / Script A makes a call to launch .NET Program B (e.g. via Process.Start())
  • .NET Program B throws an exception and it’s not immediately clear why.
  • The world ends. 

If you’re the author of program B, simply insert a call to Debugger.Launch() inside Main().  The program will halt execution and prompt you to attach the debugger.  You can then examine the conditions and fix the bug. 

Also, another thing to try is to create a handler that wraps the program’s invocation in a try/catch block, complete with an #if wrapped call to Debugger.Launch().  This allows you to connect to the crashing process without requiring lots of boilerplate code.

I wouldn’t recommend using this for production code in case you forget to remove it (add conditional #if guards and whatnot), but it’s nice to have in the toolbag nonetheless.

E.g.:

public static class AttachableRunner
{   
    public static void RunWithDebugger(Action _action)
    {
        try
        {
            _action();
        }
        catch(Exception e)
        {
            Debugger.Launch();
            throw;
        }
    }
}

This does just fine for debugging most problems. 

Note: It won’t attach properly if your program causes an unhandled exception to be thrown on a a thread other than the main thread. I’ll leave that as an exercise for the reader (*cough* AppDomain.UnhandledException can help, but won’t give you as much useful information beyond standard callstack info).

Data-Driven Testing with Python

Hello tests.  It’s been a while since I blogged about automated testing, so it’s nice to welcome an old friend back to the fold.  I’ve missed you.  I’ve recently started programming in python.  I say programming, but what I mean by that is ‘hopelessly jabbing at the keyboard and gawping as my lovingly written code explodes on being invoked’.  While I blub.

Python is a dynamic programming language.  It’s pretty dynamic in its ability to detonate in various ways, too.  There’s no compiler to sanity check my elementary mistakes like typos, or dotting onto a method that doesn’t exist.  Or accidentally comparing functions instead of values.  Or… well, you get the gist.

Anyzoom, the fundamentals of unit testing in python are very simple.  I can’t be bothered detailing how to use unittest or a test runner, as there’s nine zillion resources out there on it already. 

A Simple Setup

We’re using unittest as the framework, Nose as the runner and TeamCity Nose integration.  It worked very nicely out of the box, so I can’t complain.  Writing tests is simple, but then I wanted to make a data-driven test and…

Read more »

The Power of Boy Scouting & Git Stash

No, I’m not advocating beavering your way to several badges while wearing ill-fitting shorts, a neckerchief fastened with a woggle and risking a criminal record.  I’m talking about The Boy Scout Rule

The gist of it:

The Boy Scouts have a rule: "Always leave the campground cleaner than you found it." If you find a mess on the ground, you clean it up regardless of who might have made the mess. You intentionally improve the environment for the next group of campers. Actually the original form of that rule, written by Robert Stephenson Smyth Baden-Powell, the father of scouting, was "Try and leave this world a little better than you found it."

What if we followed a similar rule in our code: "Always check a module in cleaner than when you checked it out." No matter who the original author was, what if we always made some effort, no matter how small, to improve the module. What would be the result?

Why it matters

As a programmer, sometimes it’s doing the little things that makes everybody’s collective life a little bit easier.  There’s a lot written about the practical benefits of doing this (namely: the ability to continually deliver without having to scrabble around in a dung-heap, weeping with shame) but, for my money, the most important thing is that it makes writing code more enjoyable on a day to day basis. 

Read more »

Moving Origin Games to a different drive

I wanted to move my BF3 install to my SSD but couldn’t figure out how to do it.  Numerous posts suggested things, but none worked until I found this:

From the EA forums:

http://forum.ea.com/eaforum/posts/list/7461348.page#17730650

From SicklySquid:

“OK. I thought I would pass this along like a good Samaritan.
SO I had the problem of not being able to install BF3 Into the directory I wished. I did most of what -=ADF=-Joey said, however I could not find _installer\DISK1 or EASetup. If they were here I would of found them trust me haha. So instead of uninstalling the game through the setup I simply used the windows uninstaller. (note: I moved the game to the directory on my other HD first. Also deleted the folder in the directory origin put it in.)
After I opened origin back up and instead of it saying "download" it said "install". It gave me the "working" symbol for a minute or so. Then it recognized it in the directory on my other drive. FIXT! “

Unity3d – Profiling calls to Destroy

Destroy All … Things

I’ve recently been investigating some performance problems in our Unity3d app.  In one specific instance (running full screen on a rubbish laptop), there were numerous large performance spikes caused by the nebulous-sounding “Destroy”.  After prodding a little bit, Destroy is related to calling GameObject.Destroy().

Unfortunately, Unity3d’s profiler won’t give you any information related to what types of objects are being destroyed, and wrapping your GameObject.Destroy calls in profiler sections doesn’t help, as Unity3d defers the work for later in the frame (so the methods return nigh-on immediately).  As such, you get told “Destroy is taking x ms this frame”, and that’s about it.

Finding the culprits with the profiler

I managed to work around this limitation by (temporarily) changing all GameObject.Destroy() method calls to GameObject.DestroyImmediate() then wrapping the calls in Profiler.BeginSample() / Profiler.EndSample() pairs.

Note: If you access your unity instances in the same frame after calling destroy, this probably won’t work for you.

It was then possible to see which resources were doing the damage on the laptop.  All of our resource cleanup code was in one place, so it was trivial to do.

The temporarily instrumented code ended up looking something like this, and immediately let us know the culprits.  Note this code is just a simplified mockup, but it should give you the gist of the idea:

// centralised resource cleanup makes profiling simple
private void CleanupResources<TResource>()
{
    Profiler.BeginSample("Destroy: " + typeof(TResource).Name);
    IEnumerable<TResource> resources = FindResourceOfType(typeof(TResource));
    foreach(var resource in resources)
    {
        resource.Dispose();
    }
    Profiler.EndSample();   
}

//… and each Resource type inherits from a common base class, implementing IDisposable.
public abstract class Resource : IDisposable
{
    protected abstract void CleanupUnityResources();
   
    public void Dispose()
    {
        CleanupUnityResources();
    }
}

public class SomeResource : Resource
{
    private Mesh m_unityMesh; // gets set when resource is locked in
   
    protected override void CleanupUnityResources()
    {
        // GameObject.Destroy(m_unityMesh);
        GameObject.DestroyImmediately(m_unityMesh);
    }
}