Category Archives: gotchas

Unity3d–Threadpool Exceptions

A quickie, but something to be very wary of.  I’ve been using Unity3d of late (I recommend it – it’s a very opinionated and sometimes quirky bit of software, but it generally works well) and I was recently tasked to parallelise some CPU intensive work. 

I decided, quite reasonably, to use the built-in ThreadPool rather than doing my own explicit management of threads, mainly because the work we’re parallelising is sporadic in nature, and it’s easier to use the ThreadPool as a quick first implementation.  So far, so good.  Everything was going swimmingly, and it appeared to work as advertised.  In fact, the main implementation took less than a day.

Most .NET developers who are familiar with doing threading with the ThreadPool will know that, post .NET 1.1, if an unhandled exception occurs on a ThreadPool thread, the default behaviour of the CLR runtime is to kill the application.  This makes total sense, as you can no longer guarantee a program’s consistency once an unhandled exception occurs. 

To cut a long story short, I spent about three days debugging a very subtle bug with a 1 in 20 reproduction rate (oh threading, how I love thee).  Some methods were running and never returning a result, yet no exceptions were reported. 

Eventually I reached a point where I’d covered nearly everything in my code and was staring at a Select Is Broken situation (in that Unity3d had to be doing something weird).

Unity3d silently eats ThreadPool exceptions!  I proved this by immediately throwing an exception in my worker method and looking at the Unity3d editor for any warnings of exceptions – none was reported (at the very least, they should be sending these exceptions to the editor). 

I then wrapped my worker code in a try catch and, when I finally got the 1 in 20 case to occur, sure enough, there was an exception that was killing my application’s consistency.  So yes, I did have a threading bug in my application code, but Unity3d’s cheerful gobbling of exceptions meant the issue was hidden from me.

I’ve bugged the issue, so hopefully it’ll get fixed in future.

Note: To the people who say “you should be handling those exceptions in the first place”, I would say, “not during development”. When developing, I generally want my programs to die horribly when something goes wrong, and that is the expected behaviour for the .NET ThreadPool.  Unhandled exceptions make it clear that there’s a problem and it means the problem must be fixed.

Castle DynamicProxy2 quirks

I’ve been faffing around with Castle.DynamicProxy2 a bit lately and it’s a pretty interesting bit of kit.  Castle Dynamic Proxy (CDP) allows you to dynamically generate proxies at runtime to weave aspects of behaviour into existing types.  Aspect oriented programming is typically employed for implementing crosscutting concerns such as logging, performance measuring, raising INotifyPropertyChanged and various other types of repetitive and/or orthogonal concerns.  I’m a newbie to this stuff so I won’t say much more on AOP.

While I really like CDP, I’ve found that the documentation and tutorials (the best of which is Krzysztof Koźmic‘s excellent tutorial series) aren’t particularly explicit on how CDP achieves its effects, and sometimes these details are important.

There are two main ways of creating proxies that most developers will encounter.

CreateClassProxy

This is nearly always the first demonstrated method in tutorials.  ProxyGenerator.CreateClassProxy dynamically subclasses the target class, so if you have a class named Pogo and you call ProxyGenerator.CreateClassProxy, what you’ll get back is an instance of a subclass of Pogo (i.e. the new type is-a Pogo) that weaves in the interception behaviour via overriding methods.  This is why it is a stipulation that methods / properties must be virtual when they’re intercepted.

With class based interceptors, you cannot intercept non virtual methods because unlike Java, C# does not make methods virtual by default.  If you try to intercept a non-virtual method, nothing will happen, though mechanisms do exist to allow you to identify these situations and warn the developer (the most common example of this is that NHibernate will die on its arse if you try to use lazy loading with a non-virtual member).

CreateInterfaceProxyWithTarget

The second method is ProxyGenerator.CreateInterfaceProxyWithTarget, and it is the primary reason for writing this blog post!  CreateInterfaceProxyWithTarget does not dynamically subclass target types, it simply creates a dynamically generated class, implements the same target interface and then passes through to it.  I.e. it’s an implementation of the decorator pattern.  This has two effects, one of which is very important!

  1. You don’t need to mark your methods/properties as virtual
  2. Since it is a proxy working via decoration rather than subclassing, for the effects of the interceptor to be applied, all calls must be made on the proxy instance.  Think about it.

The most salient point is #2.  I’ll elaborate.

A worked example: Rocky Balboa

Say you have an interface called IBoxer like this:

… and you implement it like this:

If you then turn to aspect oriented programming and decide to gather statistics on punches thrown for the duration of a boxing round, it’s a reasonable assumption that you can simply proxy the IBoxer interface and intercept only the StraightLeft/StraightRight punch calls, tally them up and report the metrics (ignore whether this is a good idea to be doing this, it’s a contrived example).  On the face of it this isn’t a horrible idea.  However, it won’t work as expected.

The key here is that OneTwo() calls through to StraightLeft() and StraightRight().  Once the proxy has delegated to the decorated type it loses the ability to intercept the calls.  We can follow the call graph easily enough.  We have a reference to the proxy via the IBoxer interface.   We call “OneTwo()” on it and when the invocation proceeds, it delegates to the decorated Rocky instance.  The Rocky instance then calls StraightLeft(), StraightRight().  Both of these calls will immediately go straight to the ‘real’ implementation, bypassing the proxy.

Just as with the normal decorator pattern, the decorator (the IBoxer dynamic proxy in this case) loses its influence when the decorated object calls methods declared in its own public interface.  In this particular situation we could write an interceptor that knows to add two punches when OneTwo() is called on the proxy, but compare this approach to one using class based proxy.  If we were using a class proxy we could rest safe in the knowledge that all calls to StraightLeft() and StraightRight() will always be intercepted, as the extra stuff we’ve bolted on resides in a method override.

The results vary depending on the type of proxy we generate and the way the types are written. In hindsight it’s pretty obvious, but it still caught me out.

Testing gotchas – c# Weak References

If you ever have to test a class that uses a WeakReference, or even just have to use Weak References, be very careful.  Numerous strange-looking things can occur when Weak References are involved.

If you have even a cursory understanding of the .NET Garbage Collector (GC), you will know that it keeps track of objects.  When an object is no longer strongly referenced, the GC will potentially collect it, freeing up its resources.  This causes the object to ‘disappear’. So, if you have a strong reference to an object in your program, you are generally safe in the assumption that the object will stick around.  The GC won’t pull the carpet from under you while you’re using that object.

Weak References, on the other hand, do not stop the GC from collecting the object they refer to.  In certain circumstances, it can be advantageous to use Weak References because you do want to use/observe/whatever an object, but you don’t want to stop it from being collected.  So far so good.  All obvious stuff.

The GC moves in mysterious ways

OK you say, what’s the point in this article?  The point is that GC is extremely clever.  Almost a little too clever.  So clever it may aggressively collect objects to the point where it can mess with your head, and your tests.  This ‘problem’ can manifest itself in subtle ways — certain types of tests involving Weak References will usually pass in debug mode, but may sporadically fail in release mode.  Heads will be scratched.  Bemused gurning will commence.

Obligatory Contrived Example

Why does this occasionally fail?

If you look carefully, you may spot the problem.  Recall that I said that objects can be collected while still in scope.  After the builder reference has been passed to the GuyWhoUsesWeakRef constructor, it is no longer used anywhere.  The GuyWhoUsesWeakRef class doesn’t take a strong reference, so the moment the parameter is no longer used, that reference also gets discarded.

As a result, immediately after the new GuyWhoUsesWeakRef(builder) call, the GC figures out that the StringBuilder object we’ve created will never be used again.  After all, if the object is never used again, why not collect it as soon as possible?

In debug mode, this won’t throw a spanner in the works.  The test will pass because the GC is not aggressively collecting.  However, in release mode, the GC may well collect the StringBuilder when we fully expect it to still be alive for our Assert.That() call.

The main problem is that it won’t happen every time.  The GC is non-deterministic, so this test will pass and fail intermittently; it depends on the timing of the collection.  Coming from C++ where objects are destroyed as they exit scope, I found this somewhat bemusing.  In the context of the GC, it makes sense, though.  You just have to be careful.

The solution

The good folks at Microsoft they provided a very simple static method call to solve this particular problem; enter GC.KeepAlive.  Placing a call to GC.KeepAlive(builder) at the end of this test method will ensure that the object we’re referring to will not be collected until after the GC.KeepAlive call has been made.  Problem solved.