Category Archives: c#

Integer to binary string in c#

Python has the terribly useful functionality of bin() to do this, but what about C#?

On the rare occasion I have to monkey around with bitwise operators, I always keep this handy. It has the added bonus of printing the leading zeros – I find this very useful when writing/debugging bitwise code, as it makes it easy to line up bitmasks etc. in the immediate window / console.

public static class BinaryPrinter
{
    public static string ToBinaryString(long _long)
    {
        // have to do this or it won’t print the leading 0 values       
        return Convert.ToString(_long, 2).PadLeft(sizeof(long) * 8, ‘0’); 
    }
}

Similarly, you could write a few of these (for all of the various inbuilt types) and chuck them in extension methods.

Functional-style programming – Easy debugging!

Ever since LINQ arrived in C# land, a lot of people have been using a functional style of programming without even perhaps being conscious of it. 

I always chuckle when I read the words “functional style”, because I picture grim marches across a barren landscape, brutalist architecture and people with beards. 

One particular facet of programming in a functional style is where, rather than mutating an existing piece of data, input data is transformed / projected into something new via a pure function.  Since this style of programming limits the number of side-effects, it makes debugging a comparatively wonderful experience. 

Read more »

The wonders of Debugger.Launch()

Ever worked on a project that involved spawning new .NET processes? (as in, one [arbitrary] program launches another .NET executable)  I’ve had to do this on quite a few occasions over the years and the one thing that always saves my bacon when it comes to understanding and fixing bugs is Debugger.Launch().

A common scenario is as follows:

  • Program / Script A is launched
  • Program / Script A makes a call to launch .NET Program B (e.g. via Process.Start())
  • .NET Program B throws an exception and it’s not immediately clear why.
  • The world ends. 

If you’re the author of program B, simply insert a call to Debugger.Launch() inside Main().  The program will halt execution and prompt you to attach the debugger.  You can then examine the conditions and fix the bug. 

Also, another thing to try is to create a handler that wraps the program’s invocation in a try/catch block, complete with an #if wrapped call to Debugger.Launch().  This allows you to connect to the crashing process without requiring lots of boilerplate code.

I wouldn’t recommend using this for production code in case you forget to remove it (add conditional #if guards and whatnot), but it’s nice to have in the toolbag nonetheless.

E.g.:

public static class AttachableRunner
{   
    public static void RunWithDebugger(Action _action)
    {
        try
        {
            _action();
        }
        catch(Exception e)
        {
            Debugger.Launch();
            throw;
        }
    }
}

This does just fine for debugging most problems. 

Note: It won’t attach properly if your program causes an unhandled exception to be thrown on a a thread other than the main thread. I’ll leave that as an exercise for the reader (*cough* AppDomain.UnhandledException can help, but won’t give you as much useful information beyond standard callstack info).

Unity3d – Profiling calls to Destroy

Destroy All … Things

I’ve recently been investigating some performance problems in our Unity3d app.  In one specific instance (running full screen on a rubbish laptop), there were numerous large performance spikes caused by the nebulous-sounding “Destroy”.  After prodding a little bit, Destroy is related to calling GameObject.Destroy().

Unfortunately, Unity3d’s profiler won’t give you any information related to what types of objects are being destroyed, and wrapping your GameObject.Destroy calls in profiler sections doesn’t help, as Unity3d defers the work for later in the frame (so the methods return nigh-on immediately).  As such, you get told “Destroy is taking x ms this frame”, and that’s about it.

Finding the culprits with the profiler

I managed to work around this limitation by (temporarily) changing all GameObject.Destroy() method calls to GameObject.DestroyImmediate() then wrapping the calls in Profiler.BeginSample() / Profiler.EndSample() pairs.

Note: If you access your unity instances in the same frame after calling destroy, this probably won’t work for you.

It was then possible to see which resources were doing the damage on the laptop.  All of our resource cleanup code was in one place, so it was trivial to do.

The temporarily instrumented code ended up looking something like this, and immediately let us know the culprits.  Note this code is just a simplified mockup, but it should give you the gist of the idea:

// centralised resource cleanup makes profiling simple
private void CleanupResources<TResource>()
{
    Profiler.BeginSample("Destroy: " + typeof(TResource).Name);
    IEnumerable<TResource> resources = FindResourceOfType(typeof(TResource));
    foreach(var resource in resources)
    {
        resource.Dispose();
    }
    Profiler.EndSample();   
}

//… and each Resource type inherits from a common base class, implementing IDisposable.
public abstract class Resource : IDisposable
{
    protected abstract void CleanupUnityResources();
   
    public void Dispose()
    {
        CleanupUnityResources();
    }
}

public class SomeResource : Resource
{
    private Mesh m_unityMesh; // gets set when resource is locked in
   
    protected override void CleanupUnityResources()
    {
        // GameObject.Destroy(m_unityMesh);
        GameObject.DestroyImmediately(m_unityMesh);
    }
}

Unity3d–Threadpool Exceptions

A quickie, but something to be very wary of.  I’ve been using Unity3d of late (I recommend it – it’s a very opinionated and sometimes quirky bit of software, but it generally works well) and I was recently tasked to parallelise some CPU intensive work. 

I decided, quite reasonably, to use the built-in ThreadPool rather than doing my own explicit management of threads, mainly because the work we’re parallelising is sporadic in nature, and it’s easier to use the ThreadPool as a quick first implementation.  So far, so good.  Everything was going swimmingly, and it appeared to work as advertised.  In fact, the main implementation took less than a day.

Most .NET developers who are familiar with doing threading with the ThreadPool will know that, post .NET 1.1, if an unhandled exception occurs on a ThreadPool thread, the default behaviour of the CLR runtime is to kill the application.  This makes total sense, as you can no longer guarantee a program’s consistency once an unhandled exception occurs. 

To cut a long story short, I spent about three days debugging a very subtle bug with a 1 in 20 reproduction rate (oh threading, how I love thee).  Some methods were running and never returning a result, yet no exceptions were reported. 

Eventually I reached a point where I’d covered nearly everything in my code and was staring at a Select Is Broken situation (in that Unity3d had to be doing something weird).

Unity3d silently eats ThreadPool exceptions!  I proved this by immediately throwing an exception in my worker method and looking at the Unity3d editor for any warnings of exceptions – none was reported (at the very least, they should be sending these exceptions to the editor). 

I then wrapped my worker code in a try catch and, when I finally got the 1 in 20 case to occur, sure enough, there was an exception that was killing my application’s consistency.  So yes, I did have a threading bug in my application code, but Unity3d’s cheerful gobbling of exceptions meant the issue was hidden from me.

I’ve bugged the issue, so hopefully it’ll get fixed in future.

Note: To the people who say “you should be handling those exceptions in the first place”, I would say, “not during development”. When developing, I generally want my programs to die horribly when something goes wrong, and that is the expected behaviour for the .NET ThreadPool.  Unhandled exceptions make it clear that there’s a problem and it means the problem must be fixed.

Multiple Mocks?

This post was in response to a post on the fragmental.tw blog (the comments aren’t working for me, so I thought I’d post it here) which I read via Roy Osherove.

Basically, the author was querying Roy Osherove’s rule of thumb that stated, “more than one mock per test is harmful”.

He then goes on to provide an example where one action has two identifiable/observable results, and two mocks are used.

Figure 53b: Two Mocks (I’m so, so, sorry)

image

Read more »

Castle DynamicProxy2 quirks

I’ve been faffing around with Castle.DynamicProxy2 a bit lately and it’s a pretty interesting bit of kit.  Castle Dynamic Proxy (CDP) allows you to dynamically generate proxies at runtime to weave aspects of behaviour into existing types.  Aspect oriented programming is typically employed for implementing crosscutting concerns such as logging, performance measuring, raising INotifyPropertyChanged and various other types of repetitive and/or orthogonal concerns.  I’m a newbie to this stuff so I won’t say much more on AOP.

While I really like CDP, I’ve found that the documentation and tutorials (the best of which is Krzysztof Koźmic‘s excellent tutorial series) aren’t particularly explicit on how CDP achieves its effects, and sometimes these details are important.

There are two main ways of creating proxies that most developers will encounter.

CreateClassProxy

This is nearly always the first demonstrated method in tutorials.  ProxyGenerator.CreateClassProxy dynamically subclasses the target class, so if you have a class named Pogo and you call ProxyGenerator.CreateClassProxy, what you’ll get back is an instance of a subclass of Pogo (i.e. the new type is-a Pogo) that weaves in the interception behaviour via overriding methods.  This is why it is a stipulation that methods / properties must be virtual when they’re intercepted.

With class based interceptors, you cannot intercept non virtual methods because unlike Java, C# does not make methods virtual by default.  If you try to intercept a non-virtual method, nothing will happen, though mechanisms do exist to allow you to identify these situations and warn the developer (the most common example of this is that NHibernate will die on its arse if you try to use lazy loading with a non-virtual member).

CreateInterfaceProxyWithTarget

The second method is ProxyGenerator.CreateInterfaceProxyWithTarget, and it is the primary reason for writing this blog post!  CreateInterfaceProxyWithTarget does not dynamically subclass target types, it simply creates a dynamically generated class, implements the same target interface and then passes through to it.  I.e. it’s an implementation of the decorator pattern.  This has two effects, one of which is very important!

  1. You don’t need to mark your methods/properties as virtual
  2. Since it is a proxy working via decoration rather than subclassing, for the effects of the interceptor to be applied, all calls must be made on the proxy instance.  Think about it.

The most salient point is #2.  I’ll elaborate.

A worked example: Rocky Balboa

Say you have an interface called IBoxer like this:

… and you implement it like this:

If you then turn to aspect oriented programming and decide to gather statistics on punches thrown for the duration of a boxing round, it’s a reasonable assumption that you can simply proxy the IBoxer interface and intercept only the StraightLeft/StraightRight punch calls, tally them up and report the metrics (ignore whether this is a good idea to be doing this, it’s a contrived example).  On the face of it this isn’t a horrible idea.  However, it won’t work as expected.

The key here is that OneTwo() calls through to StraightLeft() and StraightRight().  Once the proxy has delegated to the decorated type it loses the ability to intercept the calls.  We can follow the call graph easily enough.  We have a reference to the proxy via the IBoxer interface.   We call “OneTwo()” on it and when the invocation proceeds, it delegates to the decorated Rocky instance.  The Rocky instance then calls StraightLeft(), StraightRight().  Both of these calls will immediately go straight to the ‘real’ implementation, bypassing the proxy.

Just as with the normal decorator pattern, the decorator (the IBoxer dynamic proxy in this case) loses its influence when the decorated object calls methods declared in its own public interface.  In this particular situation we could write an interceptor that knows to add two punches when OneTwo() is called on the proxy, but compare this approach to one using class based proxy.  If we were using a class proxy we could rest safe in the knowledge that all calls to StraightLeft() and StraightRight() will always be intercepted, as the extra stuff we’ve bolted on resides in a method override.

The results vary depending on the type of proxy we generate and the way the types are written. In hindsight it’s pretty obvious, but it still caught me out.

Data-driven testing tricks

It’s a fairly common occurrence — somebody wants to use NUnit’s data driven testing, but they want to vary either the action under test, or the expectation.  I.e. they’re not parametrising simple data, they’re parametrising the actions.

You cannot encode these things via normal data-driven testing (short of doing really nasty things like passing string names of methods to be invoked or using enums and a dictionary of methods) and even if you use a hackish workaround, it’s unlikely to be flexible or terse.

Test readability is paramount, so if you have some tests written in an unfamiliar style, it is very important to express the intent clearly, too.

NUnit’s data-driven testing

NUnit uses a few mechanisms to parametrise tests.  Firstly, for simple test cases, it offers the [TestCase] attribute which takes a params object[] array in its constructor.  Each argument passed to the TestCaseAttribute’s constructor is stored, ready for retrieval by the framework.  NUnit does the heavy lifting for us and casts/converts each argument to the test method’s parameter types.  Here’s an example where three ints are passed, then correctly mapped to a test method:

The main limitation here is that we can only store intrinsic types.  Strings, ints, shorts, bools etc.  We can’t new up classes or structs because .NET doesn’t allow it.  How the devil do we do something more complicated?

Passing more complicated types

It would appear we’re screwed, but fortunately, we can use the [TestCaseSource] attribute.  There are numerous options for yielding the data, and one of them is to define an IEnumerable<TestCaseData> as a public method of your test class (it works if it’s private, but since it’s accessed via reflection it’s a good idea to keep it public so that ReSharper or other tools do not flag it as unused).  You can then fill up and yield individual TestCaseData instances in the same fashion as before.  Once again, NUnit does the mapping and the heavy lifting for us.

If you do not require any of the fancy SetDescription, ExpectedException etc. stuff associated with the TestCaseData type, you can skip one piece of ceremony by simply yielding your own arbitrary type instead (i.e. change the IEnumerable<TestCaseData> to IEnumerable<MyType> and then simply yield return new MyType()).

Passing a delegate as a parameter (simple)

The simplest case is that you want to vary which methods are called.  For example, if you have multiple types implementing the same interface or multiple static methods, encoding which method to call is very simple.

Here’s an example from Stackoverflow that I answered recently where the author wanted to call one of three different static methods, each with the same signature and asserts.  The solution was to examine the method signature of the call and then use the appropriate Func<> type (Funcs and Actions are convenience delegates provided by the .NET framework).  It was then easy to parametrise the test by passing in a delegates targeting the appropriate methods.

More advanced applications

Beyond calling simple, stateless methods via delegates or passing non-intrinsic types, you can do a lot of creative and cool stuff.  For example, you could new up an instance of a type T in the test body and pass in an Action<T> to call.  The test body would create an instance of type T, then apply the action to it.  You can even go as far as expressing Act/Assert pairs via a combination of Actions and mocking frameworks.  E.g. you could say “when I call method X on the controller, I expect method Y on the model to be called”, and so forth.

The caveat is that as you do use more and more ‘creative’ types of data-driven testing, it gets less and less readable for other programmers.  Always keep checking what you’re doing and determine whether there is a better way to implement the type of testing you’re doing.  It’s easy to get carried away when applying new techniques, but it’s often the case that a more verbose but familiar pattern is a better choice.

More readable data-driven tests

When the logic of a test method remains constant but the data varies, data-driven testing is a great tool.  It allows you, the test author, to write compact code and to add new test cases rapidly.  Unfortunately, data-driven tests have a disadvantage: The inputs are often less readable.

A simple example

Let’s take an example; testing Rob Conery’s PagedList implementation.  A page is basically a slice of the data returned by a linq query.  If more data exists beyond the ‘slice’ represented by the PagedList<T> instance, its “HasNextPage” property should return true to indicate that it is available.  Now, suppose we want to test whether a particular page has a next available page.  Three things spring to mind that can influence the result: The page size, the current page index and the number of items in the list.

Here’s a quick data-driven test for HasNextPage:

As you can see, the method itself is readable, but the parametrised values fed in via the [TestCase] attribute are not.  It’s really hard to keep everything in your head and remember what each number maps to in the function, especially when the method parameter types are all identical.  If you have a list of 20 [TestCase] attributes, you start wondering what’s for your tea and forget that the second value is the (checks image) page size.  Mince.  Mince for tea.

Hmm, if only we could those TestCases more readable; something like object initializers would be ideal.

A simple trick: Subclass TestCase

My friend Hughel helped me come up with this one and it works quite well.  Simply subclass the TestCaseAttribute class and add your own properties to represent the test parameters.  It gets a little bit hairy when you have to access the Arguments array directly (especially since Attributes can be weird), but in practice, it works fine.   In most of the tests, we’re only interested in parametrising three things, so it’s simple to add them as properties.

The end result

Finally, we apply these attributes to our data-driven test, significantly improving the readability!

I would hasten to add that I don’t recommend using this approach willy-nilly — only when you have a large amount of tests that are parametrised by the same data types, causing the test cases to become hard to follow.  I’ve used the [TestCaseSource] attribute and the [TestCase] attribute a lot in the past and most of the time it’s not a problem.

Making sense of your codebase : NDepend

I was given an NDepend license to play around with (thanks to Patrick) and said I’d blog about it.  Apologies for my tardiness!

What is NDepend?

NDepend is a piece of software that allows developers to analyse and visualise their code in many interesting ways.  Here’s an ancient relic from my hard drive, revived in NDepend:

As you can see, there’s quite a lot of functionality included.  NDepend includes features that allow you to trawl your code for quality problems, identify architectural constraints that have been breached, find untested, complicated code or pinpoint changes between builds and much more.  This post will merely scratch the surface, but will hopefully provide some decent information on what NDepend can do for you and your team.

NDepend works with Visual Studio solution files and integrates with Visual Studio, meaning it’s simple to generate an NDepend project based on a solution.  You can then press the analyse button and watch NDepend do its thing — it generates a nice HTML report with various visualisations and statistics.

CQL all the way

NDepend provide a lot of pre-defined metrics out of the box, but its best feature by far (and the one on which most of its functionality is built) is the CQL query language.  The SQL / Linq-esque query language allows you to search, filter and order data in a simple fashion.  Once you get accustomed to the query language, it is simple to start bashing out meaningful queries.

You enter queries into a command window and the results are displayed instantly.  For example, you could write something like this:

At the time of writing, 82 separate metrics are available; the metrics are grouped by categories including: assembly, namespace, method etc.  If you have an idea, then you can probably express that idea via CQL.   Furthermore, your queries can be saved and reused/applied to projects as required.

To write a query, you just tap it into the query editor:

Don’t be Paralysed by Choice

While it is a hugely powerful program, I found it overwhelming to begin with.  The default analysis of your code will result in a verbose report including every metric under the sun; some of these are unarguably very useful (Cyclomatic complexity, IL nesting depth) whereas others have a more narrow purpose (suggesting which attributes should be sealed, boxing and unboxing warnings and so on).

Due to the mammoth amount of information in the report, it’s akin to pulling the FxCop lever for the first time.  You get metrics-shotgunned in the face and potentially paralysed by choice.  As a result, I chose to view the default report as a rough guide for how to use the CQL query language and to give me ideas of how I could form queries that were tailored to my needs.  There’s an excellent placemat PDF available for download, too.

I would encourage first time users to skim the main report for interesting nuggets then immediately begin to play around with the CQL query language :).  It’s really cool to think, “I wonder if…”, type a few words and immediately see the question answered.

Manual trawling versus NDepend

Back when I was a Test Engineer, I was assigned to look at an unfamiliar part of our codebase.  I decided to do a little experiment using NDepend to determine its effectiveness (we have a few licenses at work too and are looking to integrate it into our build process at some stage soon).  The test was simply this:  I would work through the code manually checking each file/type for problems, then I would do a sweep using NDepend to see if it could pick out the same problems (and potentially some others I’d missed).

The sort of things I was looking for included:

  • Big balls of mud / God classes
  • Types with the most complicated functionality (high cyclomatic complexity / code nesting level)
  • Types that are heavily used (‘arteries’ of the codebase — failure would be more costly)
  • Types that have poor coverage and high complexity

The results were good.  The majority of the problems I had identified during a manual sweep showed up near the top of the query results. It took me a few hours to manually trawl a relatively small portion of the codebase.  It took me about an hour to get to grips with the NDepend basics, write some CQL and make sense of the results.  I’d say that’s very good going considering I hadn’t used it before.

One thing to watch out for is that some manual tweaking is often required as, if you have some ugly utility classes or use open source software in source form (such as the command line argument parser class that is ubiquitous at work), these sorts of thing monopolise the results.  To get around this problem you can ignore these types, either by applying attributes to your types in code, or by adding an explicit exclude via your query (some examples of which are included below).

Sample Queries

Here’s a few examples of how to express the aforementioned concerns as queries.  Note: I’m an NDepend newbie, so there’s probably better ways to do this.

Types that have so many lines of code that it makes your head spin

Complicated nesting

Complicated methods

High complexity with poor test coverage

‘Popular’ types with poor test coverage

None of these are ‘hard’ rules — you have to play around with them while browsing your codebase (which is easy as NDepend’s CQL editor is constantly updating as you enter the query).  You may find that one project has really simple code that doesn’t even register on a solution-wide analysis, whereas another project may hog the results pane.  If you have a specific goal in mind, you can iteratively tailor the query to get what you want :).

The role of NDepend?

NDepend is not a silver bullet and doesn’t claim to be — it’s a complementary tool that can be used in addition to buddy systems, code reviews and so forth.  Having said that, the amount of information you can mine from your codebase is pretty impressive.

In terms of practical usage, we plan to integrate it into our build system to aid us in identifying potential problems, including code changes not backed by tests, architectural rules (project x is not allowed to reference project y) and general rules of thumb that should be respected (cyclomatic complexity, nesting depth and so forth).

In short, it’s well worth checking out NDepend.