Jekyll2022-10-31T00:21:53+00:00https://marksimpson82.github.io/blog/feed.xmlMark’s DevblogThoughts about development. Oh and games stuff, too.Mark SimpsonTaking notes with Obsidian2022-10-03T00:00:00+00:002022-10-03T00:00:00+00:00https://marksimpson82.github.io/blog/2022/10/03/obsidian<h2 id="how-do-you-take-notes">How do you take notes?</h2>
<p>I’ve been programming professionally since ~2008, and I’ve tried quite a few different approaches during my career.</p>
<h3 id="approach-pencil-and-paper">Approach: pencil and paper</h3>
<p>There’s something nice about low technology note-taking. I used paper for a good chunk of my time at eeGeo/WRLD3D.</p>
<p>Pros:</p>
<ul>
<li>Using paper lets you (forces you!) to step away from the keyboard for a moment</li>
<li>You can take your pad elsewhere and prod at your thoughts elsewhere</li>
<li>There’s no real substitute for sketching diagrams quickly on paper</li>
<li>You can easily draw attention to more important ideas via highlighters etc.</li>
</ul>
<p>Cons:</p>
<ul>
<li>Not searchable without a filing system, and even then it won’t be as good as grep</li>
<li>You need to file the notes somewhere if you want to keep them around</li>
<li>They take up space</li>
<li>You can’t easily copy paste via your computer</li>
<li>You can’t use hyperlinks or other embedded, digital resources</li>
</ul>
<h3 id="approach-shared-wiki">Approach: shared wiki</h3>
<p>We were forced to use a wiki at Realtime Worlds to log our work notes for the day. The idea was that if we invented & patented something novel, we’d have a trail of evidence.
Similarly, if somebody tried to patent-troll us, we’d also have evidence to refute the claim.</p>
<p>Pros:</p>
<ul>
<li>It was actually kind of nice to have a formalised log of our work / thoughts</li>
<li>It was <em>somewhat</em> searchable</li>
</ul>
<p>Cons:</p>
<ul>
<li>It was cumbersome to edit</li>
<li>The search was patchy</li>
</ul>
<p>On top of this, there were a few problems with <em>enforced & public</em> note-taking (kind of conflated with the wiki model):</p>
<ul>
<li>Having to produce one’s thoughts on demand was a burden</li>
<li>You couldn’t write honestly about things</li>
</ul>
<h3 id="approach-haphazard-txt-files">Approach: haphazard .txt files</h3>
<p>For short-term tactical work, text files aren’t a bad choice. You can jot your thoughts down, add links to read and search through it later.
I tended to just shove them in a .txt file named after the bug number, or the name of the feature I was working on.
Again, this is something I did a lot of at eeGeo/WRLD3D.</p>
<p>It wasn’t perfect, but it did have its uses.</p>
<p>Pros:</p>
<ul>
<li>You can group notes by the topic easily enough</li>
<li>It’s searchable</li>
</ul>
<p>Cons:</p>
<ul>
<li>Doesn’t support rich media</li>
<li>Without conventions & some level of care, it becomes disorganised</li>
</ul>
<p>In my case, I tended to end up with a temp directory full of notes. I’d fish out the important-looking stuff and commit it somewhere (google doc, bug tracker, commit messages, comments, etc.) and the rest would rot.</p>
<p>After a while, I’d have to switch computers and lose the rest. Not great. There’s been quite a few times where I came up with a pretty smart, novel approach to solving an interesting problem. I can’t remember the details.</p>
<h3 id="approach-using-a-tool-or-a-hybrid-model">Approach: Using a tool (or a hybrid model)</h3>
<p>I’ve been using <a href="https://obsidian.md/">Obsidian</a> since early 2022, and I’m really, really liking it.</p>
<p>While I heartily recommend Obsidian, it’s not the only note-taking app in town. <a href="https://orgmode.org/">Org Mode</a> is another much talked about tool, for example.</p>
<p>Why do I enjoy Obsidian? Well, it allows me to document anything and everything and keep to a routine.</p>
<p>Pros:</p>
<ul>
<li>It has sane defaults</li>
<li>Support for tagging & linking between notes</li>
<li>Support for templated, daily notes</li>
<li>Support for images and other media</li>
<li>It has a plugin system</li>
<li>The app itself (on Windows at least) is fairly lightweight
<ul>
<li>27MB of RAM used on my Windows machine</li>
</ul>
</li>
<li>Multi-platform support (I use it on Windows & MacOS)</li>
<li>Mobile support</li>
<li>Markdown format, so no crazy proprietary stuff
<ul>
<li>Which means you can read/write notes however you like</li>
<li>If you stop using Obsidian, your notes are in plaintext</li>
</ul>
</li>
</ul>
<p>Cons:</p>
<ul>
<li>Closed source</li>
<li>Over the ~8 months or so I’ve been using it, they’ve changed the style/layout a lot</li>
<li>Not everyone will like using a dedicated GUI app</li>
<li>If you use it professionally you have to pay $50 p/a</li>
</ul>
<h4 id="routine">Routine</h4>
<p>Obsidian helped me to reinforce a routine. When I open Obsidian, it presents today’s daily note, which looks like this:</p>
<div class="language-markdown highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gh"># Daily Notes</span>
tags: #daily
<span class="gu">## TODO</span>
<span class="p">-</span> [ ] Daily walk
<span class="p">-</span> [ ] Brush teeth at lunchtime
<span class="gu">## General</span>
</code></pre></div></div>
<p>The act of checking off items definitely helps.</p>
<p>My 2 items are currently:</p>
<ol>
<li>Since COVID-19, lockdown and WFH started, I became a bit more … torpid. I <em>always</em> want to leave the house. Even if I’m doing other exercise, it’s important to keep moving via walking.</li>
<li>When I worked in an office, I brushed my teeth after lunch. I lost that habit after the office was torpedoed. My dentist has been nagging me about it.</li>
</ol>
<p>I now face the reality of my laziness via my daily notes; an unchecked box looks very bad. It’s just a walk. It’s literally just a 90-second brushing. Those boxes tend to get checked now.</p>
<p>In addition to a daily template, Obsidian comes in really handy for noting things that I’d frequently forget.</p>
<p>For example:</p>
<h4 id="jobs-interviews--people">Jobs interviews & people</h4>
<p>I had a job interview at the end of 2021 (which I failed), and another 4 or 5 at the start of 2022. A few of those were early in the process after a few phone interviews, so I dropped out because I’d received offers.</p>
<p>After that round of interviews concluded, pre-Obsidian me would’ve:</p>
<ul>
<li>Had only a few emails and details to go on</li>
<li>Thrown out his notes (either because I formatted my PC, or because a notebook got binned)</li>
<li>Half-heartedly addressed the weaknesses I needed to look at</li>
<li>Barely remembered much about it</li>
<li>Probably not followed through</li>
</ul>
<p>Post-Obsidian me does this a lot better.</p>
<ul>
<li>From each company, I know every person I spoke to, their job titles etc.
<ul>
<li>I also can remember the people who made a good impression on me and look them up in future!</li>
</ul>
</li>
<li>I have an overview of the interview process
<ul>
<li>How many stages</li>
<li>The questions they asked me</li>
<li>The answers I gave</li>
<li>My own thoughts on how I performed</li>
</ul>
</li>
<li>Why I didn’t get the job or why I withdrew from the process</li>
<li>Or if I did get an offer, exactly what it was</li>
</ul>
<p>It’s all tied together in a neat bundle and ready to be mined. Committing these thoughts to notes doesn’t take long, and I find writing about it provides a lot of opportunities for self-reflection.</p>
<p>I had been working at my previous company for 11 years, so it was important to me that I didn’t let job-hunting details wash over me.</p>
<h4 id="work-notes">Work notes</h4>
<p>I’ve been working for Infinity Works since March 2022, and in that time I’ve had to learn new things rapidly.</p>
<p>Note-taking has been useful for organising my thoughts, collecting links and writing my own take on the things I’ve learned.</p>
<p>For example, the first gig involved <a href="https://www.databricks.com/">Databricks</a>, <a href="https://greatexpectations.io/">Great Expectations</a>, <a href="https://azure.microsoft.com/en-gb/products/devops/">Azure DevOps</a> and many other topics I’d never touched before.</p>
<p>Writing was a really useful way to figure out how much I’d learned and how far I still had to go. It helped with prioritisation on what to learn, too. If I forgot something or lost my place, it was written down.</p>
<h4 id="semi-automation-notes">Semi-automation notes</h4>
<p>Have you ever installed & configured software, but didn’t fully automate it? E.g. at work, you’d tend to have a machine image or a series of scripts to install everything just so. At home? Not so much. Too much effort for something you do once every couple of years.</p>
<p>Can you remember:</p>
<ul>
<li>Everything you need to back up before you wipe your machine?</li>
<li>The irritating Windows options you need to toggle?</li>
<li>All the programs you’ve got installed</li>
<li>The obscure configuration changes you made?</li>
</ul>
<p>It’s often said that the first step to automation is writing out a list. Notes are the sweet spot for infrequently performed tasks.</p>
<p>E.g. here’s an excerpt from my Windows install notes (automating this would be overkill because it’s complicated and rarely gets used):</p>
<div class="language-markdown highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">1.</span> Go to "Add or Remove Programs" and uninstall everything possible
<span class="p"> 1.</span> Can also use <span class="sb">`Remove-AppxPackage`</span> to do this (see below for Cortana)
<span class="p">2.</span> Go to Apps & Features -> Optional Features
<span class="p"> 1.</span> Uninstall unneeded programs
<span class="p">3.</span> Taskbar
<span class="p"> 1.</span> Disable all taskbar integrations, like people/events/weather
<span class="p"> 2.</span> Display all taskbar icons
<span class="p"> 3.</span> Don't combine taskbar entries
<span class="p"> 4.</span> Display full taskbar entries on all monitors
<span class="p">4.</span> Start menu
<span class="p"> 1.</span> Delete everything, inc. live tiles etc
<span class="p"> 2.</span> Disable web search in the start menu (I think it was <span class="p">[</span><span class="nv">this</span><span class="p">](</span><span class="sx">https://www.bennetrichter.de/en/tutorials/windows-10-disable-web-search/</span><span class="p">)</span>)
<span class="p">5.</span> Delete Cortana and other Windows Apps nonsense
</code></pre></div></div>
<p>Then an embedded powershell script to remove software components that I cribbed from other sources:</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Get-AppxPackage</span><span class="w"> </span><span class="nt">-allusers</span><span class="w"> </span><span class="nx">Microsoft.549981C3F5F10</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Remove-AppxPackage</span><span class="w">
</span><span class="nx">Get-AppxPackage</span><span class="w"> </span><span class="nt">-allusers</span><span class="w"> </span><span class="nx">Microsoft.WindowsMaps</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Remove-AppxPackage</span><span class="w">
</span><span class="nx">Get-AppxPackage</span><span class="w"> </span><span class="nt">-allusers</span><span class="w"> </span><span class="nx">Microsoft.People</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Remove-AppxPackage</span><span class="w">
</span><span class="c"># etc</span><span class="w">
</span></code></pre></div></div>
<p>Finally, I have a <a href="https://chocolatey.org/">chocolatey</a> script to install my staple programs along with a list of software that’s not available via chocolatey.</p>
<p>This sort of thing is great. I would never remember many of these tips, and I’m free to mix in scripts with instructions as and when it makes sense.</p>
<h4 id="random-things">Random things</h4>
<p>I’ve written quite a few things over the last year, and keeping the notes has been very useful.</p>
<p>For example:</p>
<ul>
<li>Boiler problems / solutions, and cost/benefit tradeoffs of fixing vs. replacing
<ul>
<li>Along with niche sites that sell parts for my 2003 boiler!</li>
<li>I saved £150, and I have a list of other parts in case it fails again</li>
</ul>
</li>
<li>How to find a good value refurbished Aeron chair</li>
<li>Laser eye surgery options</li>
<li>Mortgage</li>
<li><a href="https://rootmy.tv/">Rooting my TV</a></li>
<li>Setting up <a href="https://www.openmediavault.org/">Open Media Vault</a> & <a href="https://www.plex.tv/">Plex</a></li>
<li>Fixing my 2010 Kindle when it refused to login (Amazon support were useless, had to figure it out myself)</li>
<li>How to build a <a href="https://www.youtube.com/watch?v=l4uCRuO-Ayo">Corsi-Rosenthal Cube</a> (it was hard to find UK resources)</li>
<li>De-bloating a Lenovo tablet</li>
<li>Recording a few medical issues I’d been having, along with detailed timelines
<ul>
<li>I <em>really</em> wish I had done this in 2020 when COVID appeared! (I got long COVID, but now I can’t remember how long I was KO’ed by it)</li>
</ul>
</li>
</ul>
<p>The only downside to doing a lot of personal note-taking is that it saps the blogging energy.</p>
<h2 id="helix">Helix</h2>
<p>Oh, and I’ve also been using <a href="https://helix-editor.com/">Helix</a> to write a lot of my notes rather than Obsidian.</p>
<p>Why? Well, it’s very lightweight, accessible from the terminal (which is always open) and I felt like taking my mediocre vim skills a bit further.</p>
<p>I added the following alias to my <code class="language-plaintext highlighter-rouge">.bashrc</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">alias </span><span class="nv">today</span><span class="o">=</span><span class="s2">"cd /c/work/notes && ./today.sh"</span>
</code></pre></div></div>
<p>… which then calls this janky script (this is on Windows):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="nb">set</span> <span class="nt">-euo</span> pipefail
<span class="nb">readonly </span><span class="nv">today</span><span class="o">=</span>./daily/<span class="si">$(</span><span class="nb">date</span> +%Y<span class="si">)</span>/<span class="si">$(</span><span class="nb">date</span> +%m<span class="si">)</span>/<span class="si">$(</span><span class="nb">date</span> +%F<span class="si">)</span>.md
<span class="k">if</span> <span class="o">[[</span> <span class="o">!</span> <span class="nt">-f</span> <span class="s2">"</span><span class="nv">$today</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$today</span><span class="s2"> doesn't yet exist; creating from template..."</span>
<span class="nb">cp</span> <span class="s2">"./daily/daily_template.md"</span> <span class="s2">"</span><span class="nv">$today</span><span class="s2">"</span>
<span class="k">fi
</span>hx <span class="s2">"</span><span class="nv">$today</span><span class="s2">"</span>
</code></pre></div></div>
<p>I’m quite enjoying Helix, and it’s replaced vim for my utility editor (git commit messages, etc.).</p>
<h2 id="appendix-synchronisingbacking-up">Appendix: synchronising/backing up</h2>
<p>If you’ve got your notes on one PC, you still need to make them portable or, at the very least, back them up regularly.</p>
<p>There are various ways to do this, and many of them work with with whatever digitised method you choose.</p>
<p>Obsidian has sync as a paid feature, but there other free (or free-ish) ways of doing it:</p>
<ul>
<li>Store your notes in a shared directory with Dropbox, Google Drive, OneDrive etc.</li>
<li>Add them to version control</li>
</ul>
<p>I personally store my work-related notes in Google Drive directory and/or OneDrive depending on the environment.</p>
<p>For personal notes, I synch via a git repo. This is a little cumbersome, but I can live with it.</p>Mark SimpsonHow do you take notes?Parameterising sgen (aka the .NET Microsoft.XmlSerializer.Generator) via a .csproj PropertyGroup2021-11-04T00:00:00+00:002021-11-04T00:00:00+00:00https://marksimpson82.github.io/blog/2021/11/04/dotnet5-sgen-parameters<h2 id="the-problem">The problem</h2>
<p>If you’ve landed on this post via searching the web, you probably already know what sgen.exe is, and what the
<a href="https://docs.microsoft.com/en-us/dotnet/core/additional-tools/xml-serializer-generator"><code class="language-plaintext highlighter-rouge">Microsoft.XmlSerializer.Generator</code></a>
nuget package does.</p>
<p>You probably used to <a href="https://docs.microsoft.com/en-us/dotnet/standard/serialization/xml-serializer-generator-tool-sgen-exe">invoke sgen</a>
as part of a build script or a post-build .csproj step and parametrise it as you saw fit: e.g. passing <code class="language-plaintext highlighter-rouge">/type:MyType</code> to
limit it to a single type.</p>
<p>Sgen has historically been flaky to use (as it changed location on disk depending on which Windows SDK was installed
and some other factors), so it’s great it’s now done via a nuget package. That’s the good part.</p>
<p>The bad part is that since switching to the
<a href="https://docs.microsoft.com/en-us/dotnet/core/additional-tools/xml-serializer-generator"><code class="language-plaintext highlighter-rouge">Microsoft.XmlSerializer.Generator</code></a>
nuget package, you’re thinking: “How do I pass arguments to sgen now that it’s automagically invoked via <code class="language-plaintext highlighter-rouge">dotnet build</code>?”
and also “where is the documentation?”</p>
<p>I had the same reaction, dear reader. Porting an old C# project to <a href="https://dotnet.microsoft.com/download/dotnet/5.0">.NET 5</a>
resulted in sgen exiting with an error, as the default behaviour tries to generate serializers for every type in the
target assembly. I don’t want this behaviour for a few reasons:</p>
<ol>
<li>It generates code I don’t want or need (and a bulkier serialization assembly)</li>
<li>It fails if namespacing is required (Got an assembly containing <code class="language-plaintext highlighter-rouge">NS1.Triangle</code> & <code class="language-plaintext highlighter-rouge">NS2.Triangle</code>? Sgen will
fail unless you disambiguate the types via namespacing)</li>
</ol>
<p>We can fix both issues by simply telling sgen “hey, only generate serialization types for <code class="language-plaintext highlighter-rouge">/type:MyType</code>”. Only now we
can’t, because <code class="language-plaintext highlighter-rouge">Microsoft.XmlSerializer.Generator</code> is calling the shots rather than us directly calling sgen.</p>
<h2 id="the-answer">The answer!</h2>
<p>Blessed art thou, because it wasn’t even possible until 2019 (which seems like an oversight).
The answer is in the GitHub <a href="https://github.com/dotnet/corefx/pull/36085">Pull Request</a> that added this functionality.</p>
<p>I haven’t been able to find any official documentation, so this is all we have to go on.</p>
<ol>
<li>Open the .csproj containing the serialization types</li>
<li>Add a <code class="language-plaintext highlighter-rouge">PropertyGroup</code> section</li>
<li>Set one of the properties, using the attribute naming format of <code class="language-plaintext highlighter-rouge"><SGenParamName></code>, where “ParamName” is a parameter name gleaned from the
<a href="https://docs.microsoft.com/en-us/dotnet/standard/serialization/xml-serializer-generator-tool-sgen-exe#parameters">sgen documentation</a>
<ul>
<li>E.g. <code class="language-plaintext highlighter-rouge">type</code> would become <code class="language-plaintext highlighter-rouge"><SGenType></code></li>
</ul>
</li>
<li>Save the project & then build as normal.</li>
</ol>
<p>Here’s the example XML from the <a href="https://github.com/dotnet/corefx/pull/36085">PR</a> by <code class="language-plaintext highlighter-rouge">jiayi11</code> (thank you, kind fellow):</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><PropertyGroup></span>
<span class="nt"><SGenReferences></span>C:\myfolder\abc.dll;C:\myfolder\def.dll<span class="nt"></SGenReferences></span>
<span class="nt"><SGenTypes></span>SgenTestProgram.MyType1;SgenTestProgram.MyType2<span class="nt"></SGenTypes></span>
<span class="nt"><SGenProxyTypes></span>false<span class="nt"></SGenProxyTypes></span>
<span class="nt"><SGenVerbose></span>true<span class="nt"></SGenVerbose></span>
<span class="nt"><SGenKeyFile></span>mykey.snk<span class="nt"></SGenKeyFile></span>
<span class="nt"><SGenDelaySign></span>true<span class="nt"></SGenDelaySign></span>
<span class="nt"></PropertyGroup></span>
</code></pre></div></div>
<p>For my use-case (serialization of a single type in a single project), all I needed to add was:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><PropertyGroup></span>
<span class="nt"><SGenTypes></span>Tools.Blah.MyClass<span class="nt"></SGenTypes></span>
<span class="nt"></PropertyGroup></span>
</code></pre></div></div>
<p>That’s it!</p>
<h2 id="a-bonus-tip">A bonus tip</h2>
<p>The documentation for the
<a href="https://docs.microsoft.com/en-us/dotnet/core/additional-tools/xml-serializer-generator"><code class="language-plaintext highlighter-rouge">Microsoft.XmlSerializer.Generator</code></a>
doesn’t seem to be 100% up to date for .NET 5 (and .NET 6 is due any day now, too!)</p>
<p>If you’re running with .NET 5, rather than copying the docs that suggest:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dotnet add package Microsoft.XmlSerializer.Generator <span class="nt">-v</span> 1.0.0
</code></pre></div></div>
<p>&</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><ItemGroup></span>
<span class="nt"><DotNetCliToolReference</span> <span class="na">Include=</span><span class="s">"Microsoft.XmlSerializer.Generator"</span> <span class="na">Version=</span><span class="s">"1.0.0"</span> <span class="nt">/></span>
<span class="nt"></ItemGroup></span>
</code></pre></div></div>
<p>… you can the command & XML version to <code class="language-plaintext highlighter-rouge">=5.0.0</code> to get the latest version.</p>
<h2 id="read-on-for-some-background">Read on for some background</h2>
<p>Anyway, I’ve hopefully saved you some head-banging.</p>
<p>For the rest of you that don’t know what sgen is and are vaguely interested, here’s a short explanation (warning:
this is boring, but it’s also useful to know for anyone doing XML serialization).</p>
<h2 id="sgenexe">sgen.exe</h2>
<p>Sgen is an XML serialization code generator. For a given assembly and (optionally) a type that you want to serialize, it
generates a .dll containing the serialization code <strong>at compile-time</strong>.</p>
<p>Let’s say you have a toy project like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>project_root
├── MyApp.sln
├── MyConsoleApp
│ ├── MyConsoleApp.csproj
│ ├── Program.cs
├── MyClassLibrary
│ ├── MyClassLibrary.csproj
│ ├── NeedsXmlSerialized.cs
</code></pre></div></div>
<p>Our class <code class="language-plaintext highlighter-rouge">NeedsXmlSerialized.cs</code> needs to be serialized to Xml (or deserialized) during the course of the app run.</p>
<div class="language-c# highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">NeedsXmlSerialized</span>
<span class="p">{</span>
<span class="k">public</span> <span class="kt">int</span> <span class="n">SomeInt</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In our Program.cs, we use the <code class="language-plaintext highlighter-rouge">XmlSerializer</code> class to convert our <code class="language-plaintext highlighter-rouge">NeedsXmlSerialized</code> instance to XML:</p>
<div class="language-c# highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">Program</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">serializer</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">XmlSerializer</span><span class="p">(</span><span class="k">typeof</span><span class="p">(</span><span class="n">NeedsXmlSerialized</span><span class="p">));</span>
<span class="k">using</span><span class="p">(</span><span class="kt">var</span> <span class="n">writer</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">StreamWriter</span><span class="p">(</span><span class="s">@"C:\some\file.xml"</span><span class="p">))</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">typeToBeSerialized</span> <span class="p">=</span> <span class="k">new</span> <span class="n">NeedsXmlSerialized</span><span class="p">;</span>
<span class="n">typeToBeSerialized</span><span class="p">.</span><span class="n">SomeInt</span> <span class="p">=</span> <span class="m">5</span><span class="p">;</span>
<span class="n">serializer</span><span class="p">.</span><span class="nf">Serialize</span><span class="p">(</span><span class="n">writer</span><span class="p">,</span> <span class="n">po</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>OK, so far so good. The program works, and we’ve not even mentioned sgen yet.</p>
<h2 id="if-it-works-already-why-use-sgen">If it works already, why use sgen?</h2>
<p>The short answer is performance. If the serialization types have not yet been generated, they must be created
on-the-fly at <strong>run-time</strong>. I.e. the first time you want to serialize/deserialize your class to/from XML, there is a
constant (and large!) stall. The serialization types are then cached & re-used for the rest of the run, but the
down-payment can be considerable.</p>
<p>I’ve seen this in production and it’s not pretty, especially in pathological cases where the runtime is short-lived and
only does one bout of serialization, e.g.:</p>
<ul>
<li>The program starts up (20ms)</li>
<li>It does some CPU-bound work (300ms)</li>
<li>XML serialization types are generated on the fly (350ms)</li>
<li>Serialize to XML (20ms)</li>
<li>The program exits (5ms)</li>
</ul>
<p>In this case, we spend around half of the program’s runtime generating the serialization types each and every run. This
is scandalously wasteful. If you were to gaze at the profiling data, you’ll see a monolithic block of waste that seems
to be non-app code.</p>
<p>An even better thing to do is avoid XML serialization and do something else instead. In my case we have no choice, as
the file format is sadly non-negotiable.</p>Mark SimpsonThe problem If you’ve landed on this post via searching the web, you probably already know what sgen.exe is, and what the Microsoft.XmlSerializer.Generator nuget package does.The fundamentals of unit testing: Setup structure2020-08-10T00:00:00+00:002020-08-10T00:00:00+00:00https://marksimpson82.github.io/blog/2020/08/10/the-fundamentals-of-automated-testing-setup-structure<p>This post is <a href="/blog/2012/10/24/the-fundamentals-of-automated-testing-series.html">part of a series</a> on unit testing.</p>
<h2 id="the-importance-of-test-setup-structure">The importance of test setup structure</h2>
<p>I’ve already blogged about the importance of well-structured test setup code in this series (have a look at
<a href="/blog/2014/08/07/the-fundamentals-of-unit-testing-arrange-act-assert.html">Arrange, Act & Assert</a> and
<a href="/blog/2012/11/11/the-fundamentals-of-automated-testing-use-factories.html">Use Factories</a>). However, I’ve yet to
touch upon the pros & cons of using the built-in framework <code class="language-plaintext highlighter-rouge">setup</code> & <code class="language-plaintext highlighter-rouge">teardown</code> methods.</p>
<h3 id="setup--teardown-methods">Setup & teardown methods?</h3>
<p>If you’ve used well-known testing frameworks, you’ll likely be familiar with some variant of these. They’re optional
methods that the framework runs for you automatically before each test is executed (<code class="language-plaintext highlighter-rouge">setup</code>) and after each test is
executed (<code class="language-plaintext highlighter-rouge">teardown</code>).</p>
<p>Here’s a few example links:</p>
<ul>
<li>Python: <a href="https://docs.python.org/3/library/unittest.html"><code class="language-plaintext highlighter-rouge">unittest</code></a> –
(<a href="https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp">setup</a> /
<a href="https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown">teardown</a>)</li>
<li>C#: <a href="https://docs.nunit.org/"><code class="language-plaintext highlighter-rouge">NUnit</code></a> –
(<a href="https://docs.nunit.org/articles/nunit/writing-tests/setup-teardown/index.html">SetUp / TearDown</a>)</li>
<li>C++: <a href="https://chromium.googlesource.com/external/github.com/google/googletest/+/HEAD/googletest/docs/"><code class="language-plaintext highlighter-rouge">googletest</code></a>
– (<a href="https://chromium.googlesource.com/external/github.com/google/googletest/+/HEAD/googletest/docs/faq.md#ctorvssetup">SetUp / TearDown</a>)</li>
</ul>
<p><strong>Aside</strong>: It should be noted that many test frameworks define additional methods that might execute when a
test <code class="language-plaintext highlighter-rouge">fixture</code> (or <code class="language-plaintext highlighter-rouge">class</code>) is created/destroyed, but we’ll just focus on <code class="language-plaintext highlighter-rouge">setup</code> & <code class="language-plaintext highlighter-rouge">teardown</code> for now.</p>
<h3 id="batteries-included">Batteries included?</h3>
<p>Because these conventional methods are included in the framework, they often become our default way of doing things.</p>
<p>However, I’ve never much cared for using them and I’ll do my best to explain why.</p>
<h2 id="a-contrived-example">A contrived example</h2>
<p>Before I get going, here’s some example C# test code that uses NUnit’s <code class="language-plaintext highlighter-rouge">SetUp</code> and <code class="language-plaintext highlighter-rouge">TearDown</code> methods. We use NUnit’s
<code class="language-plaintext highlighter-rouge">SetUp</code> method to initialise the <code class="language-plaintext highlighter-rouge">BankAccount</code> with no funds available. The tests then make use of the
already-initialised <code class="language-plaintext highlighter-rouge">BankAccount</code> instance to test functionality related to depositing funds.</p>
<p>There’s also some iffy-looking static state relating to <code class="language-plaintext highlighter-rouge">AuthenticationService</code>, and we’re leaning on the <code class="language-plaintext highlighter-rouge">SetUp</code> &
<code class="language-plaintext highlighter-rouge">TearDown</code> methods to make sure the instance is in a good state.</p>
<div class="language-c# highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">BankAccountTests</span>
<span class="p">{</span>
<span class="k">static</span> <span class="n">AuthenticationService</span> <span class="n">ms_authService</span> <span class="p">=</span>
<span class="k">new</span> <span class="nf">AuthenticationService</span><span class="p">();</span>
<span class="n">BankAccount</span> <span class="n">m_account</span><span class="p">;</span>
<span class="p">[</span><span class="n">SetUp</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Init</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">const</span> <span class="kt">decimal</span> <span class="n">InitialBalance</span> <span class="p">=</span> <span class="m">0.0</span><span class="n">m</span><span class="p">;</span>
<span class="n">m_account</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BankAccount</span><span class="p">(</span><span class="n">InitialBalance</span><span class="p">);</span>
<span class="c1">// let's imagine the account must be marked as accepting deposits</span>
<span class="c1">// before we can proceed due to some security features</span>
<span class="n">m_account</span><span class="p">.</span><span class="nf">AcceptDeposits</span><span class="p">();</span>
<span class="n">ms_authService</span><span class="p">.</span><span class="nf">Begin</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">TearDown</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">Cleanup</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">ms_authService</span><span class="p">.</span><span class="nf">End</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">DepositingFundsIncreasesAccountBalance</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">m_account</span><span class="p">.</span><span class="nf">DepositFunds</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">That</span><span class="p">(</span><span class="n">m_account</span><span class="p">.</span><span class="n">FundsAvailable</span><span class="p">,</span> <span class="n">Is</span><span class="p">.</span><span class="nf">EqualTo</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">WithdrawingFundsThrowsWhenAccountBalanceIsZero</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">Assert</span><span class="p">.</span><span class="n">Throws</span><span class="p"><</span><span class="n">InsufficientFundsException</span><span class="p">>(</span>
<span class="p">()</span> <span class="p">=></span> <span class="n">m_account</span><span class="p">.</span><span class="nf">WithdrawFunds</span><span class="p">(</span><span class="m">10.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There’s multiple things I don’t like about this.</p>
<h3 id="problem-the-code-jumps-around-a-lot">Problem: the code jumps around a lot.</h3>
<p>Because framework methods are ‘automagic’ (they’re similar to
<a href="http://wiki.c2.com/?TemplateMethodPattern">Template Methods</a>, but are often called via reflection), the code no longer
reads in a straight line – it jumps around.</p>
<p>Where do you naturally begin reading the code?</p>
<ul>
<li>Do you start from an individual test?</li>
<li>Do you start from the arbitrarily named method that has the [<code class="language-plaintext highlighter-rouge">Setup</code>] attribute above it?</li>
<li>Hmm, what if the test fixture also uses a
<a href="https://docs.nunit.org/articles/nunit/writing-tests/attributes/onetimesetup.html">[<code class="language-plaintext highlighter-rouge">OneTimeSetup</code>]</a> method, too?</li>
<li>What if the fixture is derived from a base fixture that has some (or all) of these methods?</li>
</ul>
<p>It quickly becomes the testing equivalent of voodoo, and the only way to be 100% sure is to step through the code in a
debugger. This is <strong>not</strong> what we want in test code. While test code can be verbose at times, it should be as simple as
possible.</p>
<p>If you need to share code, follow the same practices as when writing production code. <a href="https://en.wikipedia.org/wiki/Composition_over_inheritance">Prefer composition over
inheritance</a> unless there’s a strong reason to do otherwise.</p>
<h3 id="problem-setupteardown-code-quickly-becomes-misleading">Problem: setup/teardown code quickly becomes misleading</h3>
<p>Adding new tests to an existing fixture is an everyday occurrence.</p>
<p>One day we decide to cover off a potential edge-case by adding a test that ensures a <code class="language-plaintext highlighter-rouge">BankAccount</code> with a positive
balance will correctly receive a deposit. Easy, right? We add the following test:</p>
<div class="language-c# highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">DepositingFundsWhenAccountBalanceIsAboveZeroIncreasesBalance</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// Arrange</span>
<span class="k">const</span> <span class="kt">int</span> <span class="n">InitialBalance</span> <span class="p">=</span> <span class="m">1000.0</span><span class="n">m</span><span class="p">;</span>
<span class="n">m_account</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BankAccount</span><span class="p">(</span><span class="n">InitialBalance</span><span class="p">);</span>
<span class="c1">// Act</span>
<span class="n">m_account</span><span class="p">.</span><span class="nf">DepositFunds</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">);</span>
<span class="c1">// Assert</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">That</span><span class="p">(</span><span class="n">m_account</span><span class="p">.</span><span class="n">FundsAvailable</span><span class="p">,</span> <span class="n">Is</span><span class="p">.</span><span class="nf">EqualTo</span><span class="p">(</span><span class="m">2000.0</span><span class="n">m</span><span class="p">));</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There’s a wrinkle, though: we realise that the <code class="language-plaintext highlighter-rouge">[Setup]</code> method has already created a <code class="language-plaintext highlighter-rouge">BankAccount</code> with a balance of
zero, and the <code class="language-plaintext highlighter-rouge">[SetUp]</code> method <em>always</em> runs. That’s no good for our test case, though. We work around it by simply
re-creating the <code class="language-plaintext highlighter-rouge">BankAccount</code>. Unfortunately, we forgot to call <code class="language-plaintext highlighter-rouge">AcceptDeposits()</code>, so the test fails.</p>
<p>Even in a simple example, the moment the <code class="language-plaintext highlighter-rouge">setup</code> code ceases to apply to <em>all</em> cases in the test fixture, things start
to get… muddy.</p>
<p>At best, you’ll apply <em>some</em> setup logic in your <code class="language-plaintext highlighter-rouge">setup</code> method only to finish it off in your specific test methods.
This is a type of two-phase initialisation. It’s confusing, especially when the initialisation code jumps around.</p>
<p>At worst, you’ll do misleading work in <code class="language-plaintext highlighter-rouge">setup</code> only to throw it away, or even introduce bugs in the test code. This
wastes time and will confuse your fellow programmers.</p>
<p>Another equally questionable solution is to split the fixture in two. We then group the sets of tests into the
appropriate fixture depending on their <code class="language-plaintext highlighter-rouge">setup</code> needs. This is very inflexible form of coupling.</p>
<h3 id="problem-setupteardown-code-can-hide-bad-habits">Problem: setup/teardown code can hide bad habits</h3>
<p>In the example above, we’re able to work-around the problems caused by static state by performing bookkeeping in our
<code class="language-plaintext highlighter-rouge">setup</code> & <code class="language-plaintext highlighter-rouge">teardown</code> methods.</p>
<p>It’s also quite common to see developers use <code class="language-plaintext highlighter-rouge">setup</code> & <code class="language-plaintext highlighter-rouge">teardown</code> methods to skirt around the fact that they’re
<a href="/blog/2012/11/02/the-fundamentals-of-automated-testing-atomic.html">hitting the file system</a> in a unit test (at
which point it’s not really a unit test: it’ll run slower & be more prone to breakages).</p>
<p>Tests are consumers of your API, and if you’re having to perform awkward state management via the use of <code class="language-plaintext highlighter-rouge">setup</code> &
<code class="language-plaintext highlighter-rouge">teardown</code> to keep things rolling, it’s often a sign that the class under test is not terribly easy to use.</p>
<h3 id="solutions">Solutions</h3>
<ol>
<li><a href="/blog/2012/11/11/the-fundamentals-of-automated-testing-use-factories.html">Use Factories</a></li>
<li><a href="http://wiki.c2.com/?ObjectMother">Object Mother</a> (read the caveats, though)</li>
<li><a href="http://www.natpryce.com/articles/000714.html">Test Data Builders</a></li>
</ol>
<p>Each of these alternatives is relatively simple, can be called directly by a test method and offers some level of
configurability. Factory methods are the easiest to get going with.</p>
<p>Want a <code class="language-plaintext highlighter-rouge">BankAccount</code> pre-configured with a set balance? No problem! Simply add a method that initialises and returns
one. Every test is responsible for its own setup. Tests are no longer coupled to a one-size-fits-all <code class="language-plaintext highlighter-rouge">setup</code> method.</p>
<p>Object Mothers & Test Data Builders require a bit more up-front investment, but they pay it back in spades via providing
canned objects with sensible defaults in a variety configurations.</p>
<p>Let’s re-write our example tests to incorporate some of these things:</p>
<div class="language-c# highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">BankAccountTests</span>
<span class="p">{</span>
<span class="k">private</span> <span class="n">BankAccount</span> <span class="nf">CreateBankAccount</span><span class="p">(</span>
<span class="kt">decimal</span> <span class="n">openingBalance</span><span class="p">=</span><span class="m">0.0</span><span class="n">m</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// we've refactored the production code to get rid of the </span>
<span class="c1">// static dependency, so AuthenticationService has gone. </span>
<span class="kt">var</span> <span class="n">account</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">BankAccount</span><span class="p">(</span><span class="n">openingBalance</span><span class="p">);</span>
<span class="n">account</span><span class="p">.</span><span class="nf">AcceptDeposits</span><span class="p">();</span>
<span class="k">return</span> <span class="n">account</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">DepositingFundsIncreasesAccountBalance</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">account</span> <span class="p">=</span> <span class="nf">CreateBankAccount</span><span class="p">();</span>
<span class="n">account</span><span class="p">.</span><span class="nf">DepositFunds</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">);</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">That</span><span class="p">(</span><span class="n">account</span><span class="p">.</span><span class="n">FundsAvailable</span><span class="p">,</span> <span class="n">Is</span><span class="p">.</span><span class="nf">EqualTo</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">WithdrawingFundsThrowsWhenAccountBalanceIsZero</span><span class="p">()</span>
<span class="p">{</span>
<span class="kt">var</span> <span class="n">account</span> <span class="p">=</span> <span class="nf">CreateBankAccount</span><span class="p">();</span>
<span class="n">Assert</span><span class="p">.</span><span class="n">Throws</span><span class="p"><</span><span class="n">InsufficientFundsException</span><span class="p">>(</span>
<span class="p">()</span> <span class="p">=></span> <span class="n">account</span><span class="p">.</span><span class="nf">WithdrawFunds</span><span class="p">(</span><span class="m">10.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">[</span><span class="n">Test</span><span class="p">]</span>
<span class="k">public</span> <span class="k">void</span>
<span class="nf">DepositingFundsWhenAccountBalanceIsAboveZeroIncreasesBalance</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// Arrange</span>
<span class="k">const</span> <span class="kt">int</span> <span class="n">OpeningBalance</span> <span class="p">=</span> <span class="m">1000.0</span><span class="n">m</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">account</span> <span class="p">=</span> <span class="nf">CreateBankAccount</span><span class="p">(</span><span class="n">OpeningBalance</span><span class="p">);</span>
<span class="c1">// Act</span>
<span class="n">account</span><span class="p">.</span><span class="nf">DepositFunds</span><span class="p">(</span><span class="m">1000.0</span><span class="n">m</span><span class="p">,</span> <span class="n">Currency</span><span class="p">.</span><span class="n">USD</span><span class="p">);</span>
<span class="c1">// Assert</span>
<span class="n">Assert</span><span class="p">.</span><span class="nf">That</span><span class="p">(</span><span class="n">m_account</span><span class="p">.</span><span class="n">FundsAvailable</span><span class="p">,</span> <span class="n">Is</span><span class="p">.</span><span class="nf">EqualTo</span><span class="p">(</span><span class="m">2000.0</span><span class="n">m</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>I personally think the refactored test code is a lot easier to understand and extend.</p>
<ul>
<li>Each test reads linearly (no more ping-ponging around to understand the execution flow – we’re now dealing with a
handful of simple factory methods).</li>
<li>Each test can piggy-back on the default setup via a factory method, or have its own specific factory method.</li>
<li>We’re no longer using <code class="language-plaintext highlighter-rouge">setup</code>/<code class="language-plaintext highlighter-rouge">teardown</code> methods as a crutch for managing dodgy static dependencies.</li>
</ul>Mark SimpsonThis post is part of a series on unit testing.The fundamentals of unit testing: Data-driven tests2020-08-05T00:00:00+00:002020-08-05T00:00:00+00:00https://marksimpson82.github.io/blog/2020/08/05/the-fundamentals-of-automated-testing-data-driven-tests<p>This post is <a href="/blog/2012/10/24/the-fundamentals-of-automated-testing-series.html">part of a series</a> on unit testing.</p>
<h2 id="dont-repeat-yourself-dry">Don’t Repeat Yourself (DRY)</h2>
<p>If you’ve been programming for more than a few years, you’ve mostly likely heard the phrase <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself"><strong>D</strong>on’t <strong>R</strong>epeat
<strong>Y</strong>ourself</a> (DRY) and have applied it to your code.</p>
<p>The main thrust of DRY is avoiding duplication across code, data, abstractions etc. The benefits include increased code
clarity, brevity and the creation of a single source of truth (it should be noted that blindly applying the DRY
principle can be harmful, but that’s for another day).</p>
<h2 id="dry-fail-in-test-code">DRY fail in test code</h2>
<p>So, that’s DRY in production code. Seems straightforward.</p>
<p>If you’ve been writing automated tests for a while, you’ve also probably noticed that it’s very easy to repeat ourselves
when writing tests, particularly when we wish to vary the test data while retaining the same test logic.</p>
<h3 id="a-simple-example">A simple example</h3>
<p>Let’s take a very simple example: We’re going to test the <code class="language-plaintext highlighter-rouge">Python</code> addition operator:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">AdditionOperatorTests</span><span class="p">(</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_add_a</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="mi">2</span> <span class="o">+</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_add_b</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="mi">2</span> <span class="o">+</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_add_c</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="mi">3</span> <span class="o">+</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">13</span><span class="p">)</span>
</code></pre></div></div>
<h4 id="problems-with-this-approach">Problems with this approach</h4>
<p>If you look at this python example, we can say a few things:</p>
<ol>
<li>
<p>The test names aren’t particularly descriptive – it feels like we’re fighting to come up with a useful name, but
can’t.</p>
</li>
<li>
<p>The test <em>logic</em> (the assert in this case) is identical across all three tests – we’ve violated the DRY principle.</p>
</li>
<li>
<p>The bulk of the test lines are boilerplate/noise. All we’re really interested in is varying the data here.</p>
</li>
</ol>
<p>For more complicated test cases (especially those that have more intricate
<a href="/blog/2014/08/07/the-fundamentals-of-unit-testing-arrange-act-assert.html">Arrange, Act & Assert</a>) logic, the
duplication soon gets out of control.</p>
<h3 id="a-solution-data-driven-testing">A solution: data-driven testing</h3>
<p>Data-driven testing is much what it sounds like – the test logic is written once, and we data-drive the test by
passing in multiple values.</p>
<p>Most languages have testing frameworks that support data-driven testing. For example, Python has the
<a href="https://ddt.readthedocs.io/en/latest/example.html">ddt</a> package. Installing it is as simple as <code class="language-plaintext highlighter-rouge">pip install ddt</code>.</p>
<p>We can now re-write our addition operator tests to use a single method and multiple test data inputs:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">unittest</span> <span class="kn">import</span> <span class="n">TestCase</span>
<span class="kn">from</span> <span class="nn">ddt</span> <span class="kn">import</span> <span class="n">ddt</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">unpack</span>
<span class="o">@</span><span class="n">ddt</span>
<span class="k">class</span> <span class="nc">AdditionOperatorTests</span><span class="p">(</span><span class="n">TestCase</span><span class="p">):</span>
<span class="o">@</span><span class="n">data</span><span class="p">(</span>
<span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span>
<span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">),</span>
<span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">13</span><span class="p">))</span>
<span class="o">@</span><span class="n">unpack</span>
<span class="k">def</span> <span class="nf">test_add</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">expected</span><span class="p">):</span>
<span class="bp">self</span><span class="p">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">a</span> <span class="o">+</span> <span class="n">b</span><span class="p">,</span> <span class="n">expected</span><span class="p">)</span>
</code></pre></div></div>
<p>It works via three special decorators:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">@ddt</code>: marks the TestCase-derived class as being data-driven.</li>
<li><code class="language-plaintext highlighter-rouge">@data</code>: contains the test data values.</li>
<li><code class="language-plaintext highlighter-rouge">@unpack</code>: (optional) automatically unpacks the <code class="language-plaintext highlighter-rouge">@data</code> contents <code class="language-plaintext highlighter-rouge">=></code> method args (if you omit this, your test will
receive a single argument and you must unpack the contents yourself).</li>
</ul>
<h4 id="wrapping-up">Wrapping up</h4>
<p>Let’s revisit the three problems we previously talked about:</p>
<ol>
<li>
<p>The awkward naming problem has disappeared.</p>
</li>
<li>
<p>The test <em>logic</em> is defined once. It’s DRY.</p>
</li>
<li>
<p>Even in this artificially simple example, the number of lines in the test scales with the amount of test <em>data</em>.
If we want to add another test, we probably just need to add a single line.</p>
</li>
</ol>Mark SimpsonThis post is part of a series on unit testing.Goodbye Wordpress, Hello Jekyll and Github Pages2020-08-02T00:00:00+00:002020-08-02T00:00:00+00:00https://marksimpson82.github.io/blog/2020/08/02/goodbye-wordpress-hello-jekyll-and-github-pages<p>I’ve been using Wordpress with <a href="https://34sp.com">34sp.com</a> hosting since the late noughties, and it’s time for a
change.</p>
<h2 id="reasons-for-switching">Reasons for switching</h2>
<p>I’ve got a few reasons for making the switch:</p>
<ol>
<li>
<p>Wordpress is a permanent security and maintenance risk.</p>
</li>
<li>
<p>Authoring posts using markdown is a lot more pleasant & predictable.</p>
</li>
<li>
<p><a href="https://34sp.com">34sp.com</a> sadly withdrew their personal hosting product and forcibly migrated me to a professional one.</p>
</li>
</ol>
<p>In short, it’s a clunk dev experience and fairly expensive.</p>
<h2 id="making-the-switch">Making the switch</h2>
<p>As with many web topics, it wasn’t particularly difficult, but did involve a certain amount of head-scratching when
mashing the various parts together.</p>
<h3 id="create-github-repos--enable-github-pages">Create github repo(s) & enable github pages</h3>
<p>If you’re new to <a href="https://pages.github.com/">github pages</a> like me, you’ll possibly get confused by the fact that there
are two types of github pages sites: <code class="language-plaintext highlighter-rouge">organisation</code> sites & <code class="language-plaintext highlighter-rouge">project</code> sites.</p>
<p>Organisation sites are limited to one per user or organisation, but you can have multiple project sites.</p>
<p>I created two repos (caveat: I’m in ‘just-make-it-work-mode’, so exercise your own judgement):</p>
<ol>
<li>
<p><a href="https://github.com/marksimpson82/marksimpson82.github.io"><code class="language-plaintext highlighter-rouge"><myusername>.github.io</code></a> for my (2007!) root website
content</p>
</li>
<li>
<p><a href="https://github.com/marksimpson82/blog"><code class="language-plaintext highlighter-rouge">blog</code></a> for my blog</p>
</li>
</ol>
<p>The former will serve content for <a href="https://defragdev.com">https://defragdev.com</a> & the latter for
<a href="https://defragdev.com/blog/">https://defragdev.com/blog/</a>. For now, we’ll leave them being served from
<a href="https://github.io">github.io</a>, though.</p>
<p>I found it a little confusing, but there’s a convention whereby if you have a user/org repo with github pages enabled,
further repos are served via subdirectory url schemes (i.e. hitting <code class="language-plaintext highlighter-rouge">yourdomain.com/X</code> will map to the contents held in
repo <code class="language-plaintext highlighter-rouge">X</code>).</p>
<p>As blog used to be served from <a href="https://defragdev.com/blog">https://defragdev.com/blog</a>, this worked fine for me.</p>
<h3 id="export-the-data-from-wordpress">Export the data from wordpress</h3>
<p>I followed a guide from <a href="https://talk.hyvor.com/">https://talk.hyvor.com/blog/migrate-from-wordpress-to-jekyll/</a>.</p>
<p>The gist:</p>
<ul>
<li>Install <a href="https://wordpress.org/plugins/jekyll-exporter/">jekyll-exporter</a> on your wordpress site</li>
<li>Enable it</li>
<li>Export the data & download it (it may take a few minutes to generate the .zip)</li>
<li>Disable it</li>
</ul>
<p>This will give you the skeleton for your new jekyll-based site.</p>
<h3 id="install-jekyll">Install jekyll</h3>
<p>I tried to install it using WSL and it didn’t go very well(tm). Eventually got it working on Windows 10 directly, but I
can’t remember the exact steps I took. If you’re using Mac or Linux, you’ll likely have an easier time.</p>
<p>You can try to build the site via:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>bundle <span class="nb">exec </span>jekyll serve
</code></pre></div></div>
<h4 id="fix-wordpress-ids-if-needed">Fix wordpress ids (if needed)</h4>
<p>If you’re like me and your ancient wordpress scheme used integer post ids & query strings, then you’re in
for a treat – it won’t work.</p>
<p>E.g. an old URL might look something like: <code class="language-plaintext highlighter-rouge">https://defragdev.com/blog/?p=50</code></p>
<p>… which generates Front Matter like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
id: 847
title: "Nvidia + G-SYNC - Black screen when alt-tabbing"
date: 2020-03-29T21:14:28+00:00
author: Mark Simpson
layout: single
guid: https://defragdev.com/blog/?p=847
permalink: /?p=847
---
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">permalink</code> doesn’t work, as it’s a query string (<code class="language-plaintext highlighter-rouge">jekyll serve</code> will fail with an obscure-looking error).</p>
<p>If you have this problem, comment out all permalinks in your posts:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#permalink: /?p=847
</code></pre></div></div>
<p>We’ll come back to this later. If there’s nothing else jiggered, the site should now build & serve locally.</p>
<h3 id="choose-an-appropriate-url-scheme">Choose an appropriate URL scheme</h3>
<p>The default URL scheme seems pretty wordy to me, and will change the URL based on whether you add or remove categories
or tags. While it does create a hierarchical navigation structure, I decided to jettison it for something simpler.</p>
<p>To do this, edit the <code class="language-plaintext highlighter-rouge">permalink</code> value in <code class="language-plaintext highlighter-rouge">_config.yml</code> and change it to something more succinct, like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>permalink: /:year/:month/:day/:title:output_ext
</code></pre></div></div>
<h3 id="set-up-the-minimal-mistakes-theme-optional">Set up the <a href="https://mmistakes.github.io/minimal-mistakes/">minimal mistakes</a> theme (optional)</h3>
<p><a href="https://mmistakes.github.io/minimal-mistakes/">minimal mistakes</a> is a nice-looking theme, and was easy to configure.</p>
<h3 id="set-up-tagcategory-support-optional">Set up tag/category support (optional)</h3>
<p>If you want to enable browsing by tags and/or categories, you’ll need to do a bit more wrangling. This was the step that
probably caused me the most grief, as I couldn’t easily understand what was built into minimal mistakes vs. the plugins
it integrated with.</p>
<p>You can either use something out-of-the-box (and potentially deal with some DRY-fail) or plump for a jekyll plugin.</p>
<p>In the end, I settled on <a href="https://github.com/jekyll/jekyll-archives">jekyll-archives</a>. However, there’s a caveat:
<a href="https://github.com/jekyll/jekyll-archives">jekyll-archives</a> is not directly supported by github pages.</p>
<p>I.e. If you go down this route, you’ll need to handle building the site contents yourself. Fret not though, as it’s
fairly easy and covered in the next point!</p>
<h3 id="set-up-github-workflows-to-build-the-site-optional">Set up github workflows to build the site (optional)</h3>
<p><strong>Note</strong>: If you’re using plugins that are 100% compatible with github pages, you can skip this step. I’m using
jekyll-archives, which is not supported.</p>
<p>The official <a href="https://jekyllrb.com/docs/continuous-integration/github-actions/#setting-up-the-action">jekyll CI example</a>
was simple to follow.</p>
<p>You’ll need to create a secret with public repo access as per the guide.</p>
<p>For reference here’s my version of
<a href="https://github.com/marksimpson82/blog/blob/master/.github/workflows/github-pages.yml">github-pages.yml</a></p>
<p>Pushing to to <code class="language-plaintext highlighter-rouge">master</code> should now trigger a site build.</p>
<p><strong>Gotcha</strong>: If you’ve not set up github pages for the repo, <em>nothing will happen</em>. You’ll see the github action is
present, but it won’t do anything.</p>
<h3 id="set-up-a-redirection-scheme-optional">Set up a redirection scheme (optional)</h3>
<p>If you’re migrating from wordpress then you’ll want to keep your old URLs alive. There are various redirect plugins
available for jekyll, so have a poke around. The front matter of each post should contain the old, exported URLs.</p>
<h4 id="the-happy-path">The happy path</h4>
<p>If your wordpress install was configured to use sensible URL schemes like <code class="language-plaintext highlighter-rouge">mysite.com/blog/why-parsnips-are-great/</code> then
you’ll have no problem – it’s just a case of editing the front matter for each post and inserting the redirects.</p>
<p><strong>Note</strong>: If you have more than a handful of posts, I’d recommend writing a one-shot script to handle this.</p>
<h4 id="the-sad-path-querystrings">The sad path (?query=strings)</h4>
<p>My blog was unfortunately using an ancient URL scheme featuring integer ids and query parameters (e.g. <code class="language-plaintext highlighter-rouge">/blog/?p=123</code>)
and this hobbles things somewhat.</p>
<p>If you try to use a redirect plugin and configure it with something like
<code class="language-plaintext highlighter-rouge">/blog/?p=123 => /2014-02-13/why-parsnips-are-great/</code>, it won’t work (it’ll crash when attempting to <code class="language-plaintext highlighter-rouge">jekyll serve</code>).</p>
<p>I went with a quick ‘n’ dirty solution yoinked from
<a href="https://github.com/danvk/danvk.github.io/commit/f99fa0d6ef808a2ba468587d3f7eab800d448f1e">danvk</a>. It’s a simple js file
that redirects via searching the current URL. It likely won’t work with bots etc. but it’ll do for now.</p>
<p>To populate it, I did the following:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">grep</span> <span class="nt">-ri</span> permalink _posts <span class="o">></span> /c/temp/guids.txt
</code></pre></div></div>
<p>Sample content:
<code class="language-plaintext highlighter-rouge">_posts/2008-11-08-assertthatpostcontent-textdoesnotcontainhello-world.md:#permalink: /?p=3</code></p>
<p>I then wrote <a href="https://gist.github.com/marksimpson82/fdc8f6915a6256391138fdc70a0a9a0b">a one-shot python script</a> to spit
out the redirect line & pasted the lot into
<a href="https://github.com/marksimpson82/blog/blob/master/redirect.js">redirect.js</a>.</p>
<h3 id="set-up-a-custom-domain-optional">Set up a custom domain (optional)</h3>
<p>Up till now, we’ve been browsing to a github pages URL. It’s time to configure your custom domain (if you have one).
I followed <a href="https://medium.com/@hossainkhan/using-custom-domain-for-github-pages-86b303d3918a">this guide</a>.</p>
<p>In your github repo, create:</p>
<ul>
<li>A CNAME file (<code class="language-plaintext highlighter-rouge">Settings</code> > <code class="language-plaintext highlighter-rouge">GitHub Pages</code> > <code class="language-plaintext highlighter-rouge">Custom Domain</code>) (e.g. mine’s set to <code class="language-plaintext highlighter-rouge">defragdev.com</code>)</li>
</ul>
<p>On your host configuration (e.g. google domains or whatever you’re using), create:</p>
<ul>
<li>A records to point to the github pages IPs</li>
<li>A CNAME for <code class="language-plaintext highlighter-rouge">www.yourdomain.com</code>, if desired</li>
</ul>
<p>DNS records can take up to 24 hours to cycle, but in practice it took an hour for mine to come through.</p>
<h4 id="automagic-confusion">Automagic confusion</h4>
<p>I stumbled due to something I mentioned earlier: I have a github org/user repo for my root site that uses github pages.
I couldn’t easily understand why <a href="https://defragdev.com/blog">https://defragdev.com/blog</a> was magically being served
without a CNAME being present.</p>
<p>This is just a convention with github pages (read the start of this post).</p>
<p>When I configured my <a href="https://github.com/marksimpson82/marksimpson82.github.io">user repo</a> with a CNAME (pointing at
<code class="language-plaintext highlighter-rouge">defragdev.com</code>), it indirectly means that project repos will be served as subdirectories – as such, my <code class="language-plaintext highlighter-rouge">blog</code> repo
is automagically served when I browse to <a href="https://defragdev.com/blog/">https://defragdev.com/blog/</a>.</p>
<p>This may or may not be what you want. It’s perfect for me, but you may want to set up a subdomain of
<code class="language-plaintext highlighter-rouge">blog.yourdomain.com</code> instead.</p>
<h3 id="enforce-https-optional">Enforce HTTPS (optional)</h3>
<p>I don’t want to serve via HTTP, so I forced HTTPS. Check the box under:</p>
<p><code class="language-plaintext highlighter-rouge">Settings</code> > <code class="language-plaintext highlighter-rouge">GitHub Pages</code> > <code class="language-plaintext highlighter-rouge">Enforce HTTPS</code></p>
<h3 id="fix-up-various-content-problems">Fix up various content problems</h3>
<p>Since I’ve been running every major version of wordpress since ~2008, the defaults have changed a lot over that time. As
a consequence, the exported content is quite changeable.</p>
<p>If you’ve written a lot of posts, you’re not going to have fun fixing up the content; in my own case it took about 4
hours to fix up my paltry 87 posts. The low number of posts and changeable formatting meant that it wasn’t worth
automating, so I went with occasional scripts, grep & targeted manually editing.</p>
<p>Some of the problems/fixes include:</p>
<h4 id="broken-internal-blog-links">Broken internal blog links</h4>
<p>While my hacky redirect solution worked, it’s obviously better if you can use the built-in Jekyll functionality to
create internal links.</p>
<p><strong>Fix</strong>: see <a href="https://jekyllrb.com/docs/liquid/tags/#linking-to-posts">Jekyll: linking to posts</a>. The <code class="language-plaintext highlighter-rouge">post_url</code>
syntax is handy, as <code class="language-plaintext highlighter-rouge">jekyll serve</code> will fail with an error message if you have a broken internal link.</p>
<h4 id="text-content-and-formatting-issues">Text content and formatting issues</h4>
<p>My blog’s content was all over the place and included a mixture of markdown, html etc.</p>
<p><strong>Fix</strong>: manual editing</p>
<h4 id="code-snippets">Code snippets</h4>
<p>Code snippets were similarly in a mixture of formats, all of which won’t play nicely with markdown.</p>
<p><strong>Fix</strong>: see <a href="https://jekyllrb.com/docs/configuration/markdown/#redcarpet">Jekyll: code snippets</a> – this involved a lot of
manual editing, but was a doddle compared to the bad old days of wordpress formatting.</p>
<h4 id="special-characters--entity-namesnumbers">Special characters & entity names/numbers</h4>
<p>These either hinder readability while editing posts (e.g. <code class="language-plaintext highlighter-rouge">&lt;</code> aka <code class="language-plaintext highlighter-rouge">&#60;</code> for <code class="language-plaintext highlighter-rouge"><</code>, <code class="language-plaintext highlighter-rouge">&gt;</code> aka <code class="language-plaintext highlighter-rouge">&#62;</code> for <code class="language-plaintext highlighter-rouge">></code>) and/or
don’t render correctly in a browser (<code class="language-plaintext highlighter-rouge"><pre></code> combined with <code class="language-plaintext highlighter-rouge">&nbsp</code>).</p>
<p>If you have a lot of content, it’d probably be wise to ignore entity names/numbers that render correctly in the
browser and focus on the legitimate content problems.</p>
<p><strong>Fix</strong>: I fixed these problems via search/replace mostly – be careful as sometimes the context means that the entity
names/numbers are required (or may require escaping or different quotes if removed).</p>
<h4 id="irrelevant-tags-like---more--">Irrelevant tags like <code class="language-plaintext highlighter-rouge"><--more--></code></h4>
<p><strong>Fix</strong>: Search & delete.</p>Mark SimpsonI’ve been using Wordpress with 34sp.com hosting since the late noughties, and it’s time for a change.Nvidia + G-SYNC - Black screen when alt-tabbing2020-03-29T21:14:28+00:002020-03-29T21:14:28+00:00https://marksimpson82.github.io/blog/2020/03/29/nvidia-g-sync-black-screen-when-alt-tabbing<p><strong>tl;dr:</strong> It seems G-SYNC related. I fixed it by installing the monitor driver, then toggling G-SYNC on/off.</p>
<p><strong>The problem</strong></p>
<p>I’ve had an irritating PC problem for a few months, hopefully this post helps folk with the same problem.</p>
<p>When entering/exiting full-screen video or alt-tabbing out of a game, the screen would go black for a few seconds - it was pretty sluggish, too.</p>
<p>It’s one of those issues where finding the right search terms took a while, and it tended to turn up irrelevant posts from years ago, or similar problems with solutions that no longer apply (e.g. “set X option in the driver” where X option no longer even exists).</p>
<p><strong>My PC</strong></p>
<ul>
<li>Windows 10 Home Premium, 64 bit</li>
<li>NVIDIA 1060 Ti 3GB (driver version: 442.59)</li>
<li>Benq XL2540</li>
</ul>
<p><strong>The solution</strong></p>
<p>First, I installed the monitor driver for my XL2540 (I never install monitor drivers, but I needed to do this or the GSync option failed to show up in the Nvidia Control Panel).</p>
<p>Then, I toggled G-Sync on, then back off. Detailed instructions:</p>
<ul>
<li>Open Nvidia Control Panel:</li>
<li>Click “Display > Set up G-SYNC”</li>
<li>Under the “1. Apply the following changes” text</li>
<li>Check the box named, “Enable G-SYNC, G-SYNC Compatible”</li>
<li>Click “Apply” (the screen should go black, then confirm everything is OK)</li>
<li>Uncheck the same box (“Enable G-SYNC, G-SYNC Compatible”)</li>
<li>Click “Apply” and confirm once again.</li>
</ul>
<p>Thanks to Ethan Snider for his <a href="https://www.youtube.com/watch?v=m0T0Oln6khk&lc=UgwnxpFEgo8lJnkTDHZ4AaABAg">comment</a> on a youtube video for pointing me in the right direction.</p>Mark Simpsontl;dr: It seems G-SYNC related. I fixed it by installing the monitor driver, then toggling G-SYNC on/off.Jupyter Notebook to clean(ish) HTML2019-10-17T00:12:05+00:002019-10-17T00:12:05+00:00https://marksimpson82.github.io/blog/2019/10/17/jupyter-notebook-to-cleanish-html<p>I recently polled <a href="https://reddit.com/r/competitiveoverwatch">/r/competitiveoverwatch</a> (aka /r/cow) users for their thoughts on <a href="https://playoverwatch.com/en-gb/">Overwatch</a> with regards to fun and balance. It was well-received on reddit, so I thought I’d do a quick write-up on the pros & cons of using this approach.</p>
<p>You can <a href="https://defragdev.com/overwatch_surveys/2019_09/">read the survey</a> and also check out the <a href="https://github.com/marksimpson82/overwatch_survey">code on github</a>.</p>
<h2 id="overview">Overview</h2>
<p>The survey was created with <a href="https://www.google.com/forms/about/">Google Forms</a>, which produces a raw results <a href="https://www.google.com/sheets/about/">Google Sheets</a> spreadsheet. To analyse the data and generate the graphs, I used <a href="https://pandas.pydata.org/">pandas</a> and <a href="https://seaborn.pydata.org/">seaborn</a> in conjunction with <a href="https://jupyter.org/">Jypyter Notebook</a>, along with <a href="https://jekyllrb.com/">Jekyll</a> to generate fairly clean HTML & CSS.</p>
<p>Most of the pertinent commands are in one place: <a href="https://github.com/marksimpson82/overwatch_survey/blob/master/run_all.sh">run_all.sh</a></p>
<h3 id="step-1-create-the-google-forms-survey">Step 1: Create the Google Forms survey</h3>
<p>This was the most tedious part. While Google Forms is flexible, it also feels fairly clunky. On top of the general questions, I had to create 3 sets of questions per each Overwatch hero alongside the hero’s image:</p>
<ol>
<li>Are they fun to play as?</li>
<li>Are they fun to play against?</li>
<li>Are they balanced?</li>
</ol>
<p>If there was an easy way of exporting a survey’s raw source (maybe JSON or XML), generating additional templated questions via a script and then re-importing it, I couldn’t see it.</p>
<p>Instead, I created a self-contained Google Form that had the image & the three questions. I then imported this into the parent form, renamed the fields and updated the image… for … all … 31 … heroes.</p>
<p>It was boring, but only took an hour of grunt work.</p>
<h3 id="step-2-export-the-responses-from-google-sheets">Step 2: Export the responses from Google Sheets</h3>
<p>After running the survey, I had a great response: 1200 players took the time to answer it.</p>
<p>Each Google Form has an associated Google Sheets results sheet. It’s simple to export to CSV.</p>
<h3 id="step-3-create-a-more-queryable-data-format">Step 3: Create a more queryable data format</h3>
<p>The original results .csv is a flat spreadsheet with 90+ uniquely-named columns like, “<span data-sheets-value="{"1":2,"2":"I enjoy playing against Ashe"}">I enjoy playing against Ashe</span>”, and a rating between 1.0 & 5.0. This is pretty horrible for querying results, so I transformed the data into a more database-esque format: a single general responses table with a primary key, plus a hero responses table with a foreign key, hero name, response type [“playing_as”, “playing_against”, “balance” and “value”].</p>
<p>This is the sort of quick hackery that is a <a href="https://github.com/marksimpson82/overwatch_survey/blob/master/overwatch_survey/split_into_tables.py">joy to do</a> in Python.</p>
<h3 id="step-4-create-the-notebook">Step 4: Create the notebook</h3>
<p>Not much to say here. Pandas & Seaborn takes a bit of getting used to and there were a few transformations that I couldn’t figure out using idiomatic Pandas code, but the development process was fairly painless.</p>
<p>I will say that there’s a few practices that I found useful, though:</p>
<ul>
<li>Avoid file-scoped operations. I tended to wrap my data crunching & graphing operations in functions, and avoided mutating data as much as possible.</li>
<li>Avoid using file-scoped variables as much as possible (it’s easy to rely on or mutate something that will have side-effects).</li>
<li>Get in the habit of clearing all output & running all notebook cells.</li>
<li>If you’re writing something that is useful to a reader, put it in a Markdown cell.</li>
</ul>
<p>If I were creating a larger project with Notebook, I’d probably start making python modules & calling into them rather than spamming them into the main Notebook page. I.e. don’t fall into the trap of treating Notebook files significantly differently to the usual way of writing Python code.</p>
<h3 id="step-5-strip-unwanted-elements">Step 5: Strip unwanted elements</h3>
<p>I didn’t want to have the code front and centre, as the survey results were not intended for a technical audience - the Python would just get in the way. I tried writing a template for nbconvert to strip out the Python code, but couldn’t get it working. Instead, I just wrote a quick ‘n’ dirty Python regex.</p>
<h3 id="step-6-use-nbconvert--to-markdown">Step 6: Use nbconvert -to markdown</h3>
<p>Why Markdown? Well, I tried nbconvert -to html. While the results look pretty good at a glance, it generates a page with bloated, inline CSS (hundreds and hundreds of lines) along with base64-encoded images that are directly embedded in the HTML.</p>
<p>While this is great for a drag ‘n’ drop experience and for sharing intermediate results (there’s no worrying about forgetting to include assets), it’s not something I would be comfortable serving from my website.</p>
<p>The generated HTML was ~750KB, whereas a markdown variant came in under 270KB!</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>jupyter nbconvert <span class="nt">--to</span> markdown <span class="se">\</span>
overwatch_survey/analysis.ipynb <span class="nt">--output-dir</span> static_site
</code></pre></div></div>
<p>OK, so we have fairly clean Markdown, but now what?</p>
<h3 id="step-7-use-jekyll-to-generate-html">Step 7: Use Jekyll to generate HTML</h3>
<p>Doing any kind of webdev install on Windows is a pain, but … oh well. Jekyll is a solid choice and it works well. It also generates much better HTML for my purposes than nbconvert.<span class="pl-s"><span class="pl-pds"><br /> </span></span></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">(</span><span class="nb">cd </span>static_site <span class="o">&&</span> bundle <span class="nb">exec </span>jekyll serve<span class="o">)</span>
</code></pre></div></div>
<p>That’s all I have to say about that.</p>Mark SimpsonI recently polled /r/competitiveoverwatch (aka /r/cow) users for their thoughts on Overwatch with regards to fun and balance. It was well-received on reddit, so I thought I’d do a quick write-up on the pros & cons of using this approach.MS Wheel Mouse Optical Redux (August 2018)2018-08-22T16:46:54+00:002018-08-22T16:46:54+00:00https://marksimpson82.github.io/blog/2018/08/22/ms-wheel-mouse-optical-redux-august-2018<p>Back on the WMO train again. I was using Sweetlow’s signed driver, but it stopped working again, probably due to
Windows updates. You can find the old guide
<a href="/blog/2017/05/12/ms-wheel-mouse-optical-redux-wmo-1-1-windows-10-x64.html">here</a>.</p>
<p>I’m on Windows 10 Home 64-bit, version 1803. Here’s how to get it working at the time of writing.</p>
<p>The main google result is not the actual official Sweetlow post. It links to a thread on the overclock.net forums that was not made by Sweetlow. Instead, follow the instructions on Sweetlow’s official (and up to date!) <a href="https://www.overclock.net/forum/375-mice/1589644-usb-mouse-hard-overclocking-2000-hz.html">post</a>.</p>
<p>Unfortunately, the ‘vanilla’ signed driver no longer works for me, but I would recommend trying the main instructions first.</p>
<blockquote>
<p>I had to use a workaround that is covered in his post. Specifically, the part where he says:</p>
<ol>
<li>If you have EHCI (USB2.0) Controller only on version x64 1703+ or any controller on version 1803+ use these drivers and (Test Mode or atsiv method with non Test Mode)</li>
</ol>
</blockquote>
<p>I haven’t had any luck with the atsiv method, but the test mode suggestion worked (after a bit of fumbling around).</p>
<p>Here are explicit steps on how to do this workaround. As ever, I will caveat this by saying it may not work for you.</p>
<p><strong>Backup your files</strong></p>
<ul>
<li>Backup the following files:
<ul>
<li>%systemroot%\system32\drivers\usbport.sys</li>
<li>%systemroot%\system32\drivers\usbxhci.sys</li>
</ul>
</li>
</ul>
<p><strong>Enable test mode to allow unsigned drivers</strong></p>
<ul>
<li>Open a cmd prompt with admin privileges and type the following commands
<ul>
<li><code class="language-plaintext highlighter-rouge">bcdedit -set loadoptions DISABLE_INTEGRITY_CHECKS</code></li>
<li><code class="language-plaintext highlighter-rouge">bcdedit -set TESTSIGNING ON</code></li>
<li>Reboot</li>
</ul>
</li>
</ul>
<p><strong>Finally, install the driver</strong></p>
<p>We’re going to download the official Sweetlow package (which contains the installer and versions of the driver) and replace the official driver with a patched version. We will then use the official installer to install a patched driver.</p>
<ol>
<li>Download & unzip <a href="https://www.overclock.net/attachments/45829" rel="nofollow">https://www.overclock.net/attachments/45829</a> (you need an overclock.net account to do this) to a directory called “official”</li>
<li>Download & unzip <a href="https://github.com/LordOfMice/hidusbf/blob/master/hidusbfn.zip" rel="nofollow">https://github.com/LordOfMice/hidusbf/blob/master/hidusbfn.zip</a> to a temp directory called “patch”</li>
<li>Navigate to “patch”</li>
<li>Copy the DRIVER\AMD64\1khz\hidusbf.sys file</li>
<li>Navigate to “official”</li>
<li>Replace its DRIVER\AMD64\hidusbf.sys + DRIVER\AMD64\1khz\hidusbf.sys with it (I suspect the installer uses the first of these, but I haven’t checked for sure, so replace both)</li>
<li>Still in “official”, run setup.exe</li>
<li>Check the “Filter On Device” box</li>
<li>Change the rate to 1000hz</li>
<li>Click the “Install Service” button</li>
<li>Click the “Restart” button</li>
<li>Close setup.exe</li>
<li>Open mouserate.exe (or browse to <a href="https://zowie.benq.com/en-eu/support/mouse-rate-checker.html" rel="nofollow">https://zowie.benq.com/en-eu/support/mouse-rate-checker.html</a>) and check your hz</li>
</ol>
<p>If that didn’t work, reboot. If you mess it up and your mouse stops working, simply go to device manager, uninstall the WMO via remove device, then unplug it before plugging it back in. You’re then OK to try again.</p>Mark SimpsonBack on the WMO train again. I was using Sweetlow’s signed driver, but it stopped working again, probably due to Windows updates. You can find the old guide here.Focused commits (also: git add -p)2018-04-06T00:10:28+00:002018-04-06T00:10:28+00:00https://marksimpson82.github.io/blog/2018/04/06/focused-commits-also-git-add-p<p>Many people use version control as a bucket for their stuff. They commit and merge in some shape or form, and it gets shared with their colleagues. Everyone moves on with their lives. Fire & forget sucks when using distributed version control systems.</p>
<p>The trouble with this approach is that it gets complicated to track two simple things:</p>
<ol>
<li>When was a change introduced?</li>
<li>What is the purpose of the change?</li>
</ol>
<p>How do we use git in a way that makes it easy to track both <em>when</em> a change was introduced, and the <em>intent</em> of the change?</p>
<h2 id="the-when">The when</h2>
<p>The first point (the what) has been written about a lot. The gist of the workflow I use is that constantly rebasing when pulling, and squashing any merges on their way back to master helps a great deal.</p>
<p>I.e. when you’re working on a feature branch and have local commits that haven’t been pushed, use the following to avoid noisy merge commits:</p>
<p><code class="language-plaintext highlighter-rouge">git pull origin mybranch --rebase</code></p>
<p>To keep your own (non-shared) branch up to date with master, but without creating merge commits, periodically do something like:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git pull origin master
git checkout mybranch
git rebase master
</code></pre></div></div>
<p>(Small tip: if you are pushing your own personal branch after rebasing, you can be sure not to nuke anything important by employing <a href="https://developer.atlassian.com/blog/2015/04/force-with-lease/">git push -force-with-lease</a> rather than git push -force).</p>
<p>Then when you’re ready to merge back to master:``</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git checkout master
git merge mybranch <span class="nt">--squash</span>
git commit
</code></pre></div></div>
<p>There are many advantages:</p>
<ul>
<li>It keeps the history linear</li>
<li>There’s no merge noise</li>
<li>It groups related commits</li>
<li>Reverting a feature merge is easy (i.e. you can just revert the squashed merge)</li>
<li>The default commit message will contain the hashes for all squashed commits, so fine-grained operations & the intent for all of these changes are still there.</li>
<li>… and arguably best of all, you can trivially figure out which commits introduced bugs when QA says, “hey, CI build 1923’s frob widget is misbehaving” (when commits are interleaved from a long-running branch, the offending commit may actually show up in the history as being months old).</li>
</ul>
<p>The only time I deviate from this pattern is when I’m suffering a lot of merge pain - standard rebasing means merging up your <em>individual</em> commits, so it can mean a lot of repeated work needs to be done in areas where merge conflicts frequently occur. You can interactively rebase & squash your branch’s commits to reduce the number of commits that require merging or use <a href="https://git-scm.com/docs/git-rerere">git rerere</a>, but it’s fairly complicated vs. the usual workflow.</p>
<h2 id="the-why">The why</h2>
<p>OK, so that’s the when taken care of. What about the <em>intent</em> behind the code, or a particular commit? This one’s even easier to solve, it just requires some better commit discipline.</p>
<h4 id="committing-a-crime">Committing a crime</h4>
<p>Here’s a quick scenario: A developer, Eddie, has been tasked with improving the performance of an application. Eddie profiles the code and figures out some areas they think need improvements. They make the changes, profile the app and find it’s now 20% faster. Great! Oh, and they also reformatted some source files, made a number of variable names compliant with the company’s coding standards, upversioned a header-only library, refactored some code_and_ added a few missing pieces of dev functionality that made the app easier to work with for profiling.</p>
<p>All told, 45 files have been touched, and 5000 lines of code changed (mainly due to the formatting and library churn). All of these changes are made in one commit, with the a message saying “improving performance of app”.</p>
<p>Four months later, a few bugs are discovered in a seemingly unrelated area of the app, and are traced back to this commit. Another programmer, Irene, is tasked with fixing it.</p>
<h4 id="playing-detective">Playing detective</h4>
<p>Irene loads up the offending commit on github, and immediately gets irked when she spots the “change too large to display” message. At this point, Irene’s day is much worse than it should’ve been.</p>
<p>Was it the formatting changes? I mean, it’s unlikely, but sometimes people make changes by hand and fat-finger a key or transpose something important. The simplest changes can be the most dangerous (think of the number of [ya, yb, yc, xw] errors static analysis tools find…)</p>
<p>Was it the refactoring? It touched a lot of files. Hard to say without reading the code or just reverting them, then re-applying the other changes.</p>
<p>Was it the library upgrade? Again, hard to say. Even if it is, the commit message didn’t say if the new version was added for performance reasons, so perhaps Irene will need to re-profile the app to be sure.</p>
<p>Was it the new functionality? … and so on. It’s really annoying.</p>
<h4 id="better-commits">Better commit(s)</h4>
<p>VCS commits are like code - written once and read thousands of times later (especially in complicated, high-traffic areas of code). They can be precise and explain the background behind the change, or they can be unfocused while burying the reason for making the change in the first place.</p>
<p>While we’re all busy (yes, I make numerous lapses of judgement, too…), there are simple ways you can try to be a good citizen.</p>
<p>Firstly, make your commits as concise as possible. If fixing a bug requires changing precisely two lines, then please change those lines, <em>but nothing more.</em> If your bugfix isn’t resolving all instances of the problem, the next programmer that picks it up has two lines to start pondering. Two! Not 5000. Huge difference.</p>
<p>Secondly, make your commit messages as thoughtful as possible. By this I mean “how can I make the reader understand the change I’ve just made, as well as the context?” There are numerous things to talk about. Off the top of my head:</p>
<ul>
<li>Why is the change being made?</li>
<li>How does it work?</li>
<li>If it’s a complicated change, can you describe the program’s state and how this change helps?</li>
<li>Are there any caveats?</li>
<li>Where did the solution come from, is there any prior art?</li>
<li>Why solve it in this particular way when <other way=""> seems more appropriate?</other></li>
<li>Is anything else suffering from the same problem?</li>
<li>Is it a hack that needs to be properly fixed in future?</li>
<li>Can the change be reverted when library X we’re using ships a fix instead?</li>
<li>Are there any bug numbers or related commits worth referencing?</li>
</ul>
<p>Thirdly, as an addendum to #1: do not include orthogonal changes in a single commit! If you want to change the formatting of a file please do so, but not while also changing its functionality, fixing a bug and adding a feature. Do one, then the others later.</p>
<h4 id="git-add--p">git add -p</h4>
<p>If you find yourself noodling away like I do (“ooh, I’ll just refactor this while I’m here”), it’s easy to find yourself in a situation where you want to make a focused bugfix commit, but the file also has unrelated formatting changes. With a bit of practice, you can use git add -p. The patch (-p) flag allows you to recursively split the file, then stage sections of it. You can then pick and choose which parts to stage & commit! Great.</p>
<p>Certain IDEs also have VCS integration where you can select the hunks to stage, too.</p>
<p>Nothing’s ever free, so when using patch mode, take care that your changes don’t mix & match too much, or you may accidentally take part of change A and roll it in with change B (e.g. you renamed a variable, and failed to notice it crept into a prior commit).</p>
<p>If you’re making large changes, it’s better to just do the work later.</p>Mark SimpsonMany people use version control as a bucket for their stuff. They commit and merge in some shape or form, and it gets shared with their colleagues. Everyone moves on with their lives. Fire & forget sucks when using distributed version control systems.MS Wheel Mouse Optical Redux (WMO 1.1 & Windows 10 x64)2017-05-12T01:14:20+00:002017-05-12T01:14:20+00:00https://marksimpson82.github.io/blog/2017/05/12/ms-wheel-mouse-optical-redux-wmo-1-1-windows-10-x64<p>Back in 2011, I wrote a
<a href="/blog/2011/07/15/ms-wheel-mouse-optical-wmo-1-1.html">fairly gushing post</a> about the Microsoft WMO 1.1/1.1a.
It’s a great mouse. It really is.</p>
<p><img style="margin: 10px 0px;" src="https://defragdev.com/blog/images/2011/07/image1.png" width="240" height="240" /></p>
<h2 id="the-end-of-my-wmo">The end of my WMO</h2>
<p>Sadly, in late 2016, the venerable WMO gave up the ghost – its death certificate listed a faulty cable. Basically, I’d be playing Overwatch and it’d randomly disconnect, then reconnect. I’m not a technician, so I binned it.</p>
<p>I then bought a Steelseries Rival 100, as I needed a stopgap mouse. The Rival 100 is a decent mouse, but its shape doesn’t agree with me (it’s too narrow), and it’s a touch too heavy for my tastes.</p>
<h2 id="finding-a-replacement-mouse-model">Finding a replacement mouse model</h2>
<p>I decided to try and track down a decent WMO replacement. After a bit of a hunt, I couldn’t find anything that’s 100% suitable; some people recommend Zowies like the EC2-A or FK1, and others like the Razer DeathAdder Chroma.</p>
<p>It’s tough to track down accurate information regarding the unpackaged weight of the mouse, too. The WMO weighed in at ~80g, and in my opinion is better for it.</p>
<p>Furthermore, I don’t want a mouse with stupid cloud software (S3 is down? Well, not sure how to load your mouse settings man!), and I’m not really a fan of spending £60 on a mouse in what is essentially a blind purchase. I might love it, but I might hate it.</p>
<p>If you can walk into a decently-sized PC store and try out a load of mice, I’d recommend just forgetting about the WMO and finding something new.</p>
<h2 id="however-if-youre-stubborn">However, if you’re stubborn…</h2>
<p>In the end, I decided to buy another WMO. They’ve long since been discontinued, so the main options nowadays are ebay and second-hand sites.</p>
<p>Personally, I bought some refurbished ones, plus a new one that was still in its (rather worn) box – it’d been sat on a shelf for 10 years, ha! I’d recommend going this route – there’s plenty of obscure product numbers & sites listing good/bad sensors, so you can do a bit of googling based on the seller’s pictures. It’s not an exact science, but the mice tend to be so cheap that it doesn’t matter if you buy a dud or two. My total outlay was about £25 for 3 mice. The two I’ve tested have the same product number of X802382. Both work fine.</p>
<p>Finally, <a href="http://www.ebay.com/itm/Microsoft-Wheel-Mouse-Optical-WMO-Steelseries-MOD-100-NEW-5-Colors-/121179666270?pt=LH_DefaultDomain_0&var=&hash=item61d44f5bbb&rmvSB=true">modded WMOs</a> from a specific seller on ebay are available, and the seller has a good reputation, so he’s perhaps worth a look.</p>
<h2 id="overclocking--harder-than-it-used-to-be">Overclocking – harder than it used to be</h2>
<p>I briefly touched on this in my previous post from 2011: if you don’t overclock the polling rate, the mouse performs worse. Depending on your sensitivity settings, this can be a deal-breaker and render it unusable.</p>
<p>The tl;dr here is that the WMO (and its stablemates) offers “perfect control” to a speed up to 1m/s, but exhibits negative acceleration beyond this speed <em><a href="http://www.overclock.net/t/1597441/digitally-signed-sweetlow-1000hz-mouse-driver">unless you overclock the mouse polling rate</a></em>. I won’t warble on about the nuts and bolts of this too much, as there’s a great <a href="http://www.esreality.com/?a=longpost&id=1265679&page=4">esreality article</a> about it that explains it better than I can.</p>
<p>What I will say is that, if you’re a high sensitivity gamer and don’t fling the mouse around a lot, this may not be a significant issue. However, I play Overwatch with an eDPI of 3.1k (450 CPI * 7 sensitivity) which equates to roughly 43 cm for a 360. This is on the lower sensitivity side of things for OW. Hitting a 180 means travelling ~20cm across the pad. Unfortunately, I <em>easily</em> hit the negative acceleration threshold @ 125hz, and my 180 degree turn looks more like a 100 degree bout of confusion.</p>
<p>I’d had overclocking working with Windows 10 x64 previously using an unsigned driver in test mode, but for whatever reason, I just couldn’t get it to hit better than 125hz using <a href="http://www.overclock.net/t/1597441/digitally-signed-sweetlow-1000hz-mouse-driver">Sweetlow’s new signed driver</a>. It would downclock OK (so I could run it at 30hz), so I just assumed I’d got a dud mouse since it wouldn’t go past the default settings. This was not the case.</p>
<h2 id="the-state-of-play-with-the-community-created-driver">The state of play with the community-created driver</h2>
<p>The guy that wrote the driver has showed a lot of ingenuity, and the community came together to raise the funds to sign the driver. The downside is that since it’s niche bit of kit, so getting help relies on a handful of folk. The readme is a little confusing and you have to download the files from a forum’s attachments and/or work your way through multiple broken links (sorry, not going to take point here as these files change frequently).</p>
<p>There’s also a huge thread full of “HLAP! It won’t work!” posts, and I don’t think there’s an FAQ, so you have to sift through 50+ page threads full of information that is usually not terribly relevant to your particular issue. I had almost given up when I managed to get it working!</p>
<p>So here’s the current state of affairs (May 2017):</p>
<ul>
<li>A signed driver exists which means you shouldn’t need to install an unsigned driver (this is good!)</li>
<li>Its certificate (which must be renewed yearly, I believe) expired or is about to expire (this is bad!)</li>
<li>A recent Windows 10 Content Creators update broke the signed driver on Windows 10 x64 (this is also bad!)</li>
<li>It will <em>probably</em> work for you <em>unless</em> you’re on Windows 10 x64 with the latest updates.</li>
<li>Various USB issues exist (haven’t personally run into this, but there’s lots of chat about disabling secure boot, xHCI and various other bits of hand-waving going on).</li>
<li>There’s also an alternative overclock which involves USB 3.0, but I’ve not touched that one.</li>
</ul>
<p>Sweetlow is doing a great job considering it’s just a side hobby for him, and he’s helping a lot of people. I thought I could help by detailing my problem & solution here.</p>
<h2 id="may-2017-windows-10-x64">May 2017, Windows 10 x64</h2>
<p>I will preface this by saying it’s not a great solution. An unsigned driver is involved, just like olden times. Sweetlow has said he’ll roll these fixes into the new signed driver. In the meantime, here’s how I got it working.</p>
<p>Note that this involves running windows with some system settings changed. Unless you’re comfortable with this, don’t do it.</p>
<p>I’ve also heard rumblings about some games viewing this as dodgy (from an anti-cheat perspective), but haven’t confirmed it. I ran all games for years with exactly the same method without issue.</p>
<ul>
<li>Backup the following files:
<ul>
<li>%systemroot%\system32\drivers\usbport.sys</li>
<li>%systemroot%\system32\drivers\usbxhci.sys</li>
</ul>
</li>
<li>Create an account on overclock.net</li>
<li>Download the <a href="http://www.overclock.net/t/1597441/digitally-signed-sweetlow-1000hz-mouse-driver">signed driver</a> and unzip it somewhere</li>
<li>Download the updated (non-signed) hidusbf.sys file from <a href="http://www.overclock.net/t/1597441/digitally-signed-sweetlow-1000hz-mouse-driver/480#post_25977633">this post</a></li>
<li>Replace the hidusbf.sys in the signed driver directory (DRIVER/AMD64/hidusbf.sys) with the one you just grabbed</li>
<li>Open a cmd prompt with admin privileges and type the following commands
<ul>
<li><code class="language-plaintext highlighter-rouge">bcdedit -set loadoptions DISABLE_INTEGRITY_CHECKS</code></li>
<li><code class="language-plaintext highlighter-rouge">bcdedit -set TESTSIGNING ON</code></li>
</ul>
</li>
<li>Reboot</li>
<li>Browse to the signed driver directory</li>
<li>Right click HIDUSBF.INF and choose “install”</li>
<li>Run Setup.exe</li>
<li>Check the “filter” box</li>
<li>Set the polling rate to 1000hz or whatever you want</li>
<li>Click “install service”</li>
<li>Click “restart” (the button in the setup.exe, not restart your PC!)</li>
<li><a href="http://zowie.benq.com/en/support/mouse-rate-checker.html">Check your mouserate</a> – it should be north of 125hz</li>
</ul>
<p>Thanks to sweetlow and co for this bit of magic.</p>
<h2 id="result-1000hz">Result: 1000hz</h2>
<p>My WMO is running at 1000hz again, and the perfect control speed is noticeably improved; it is now ~1.5m/s, which is more difficult to hit at my sensitivity, but certainly possible. If I really whip the mouse into a 180, the uneven turning is still evident, though much less pronounced. I’ll see how badly this affects my game and either persevere, or get a new mouse.</p>
<p>The WMO has its flaws and isn’t appropriate for everyone. If you play with a similar or even lower sensitivity than me, I would recommend looking into a more modern mouse, as the WMO won’t keep up. <a href="https://www.youtube.com/watch?v=74X-irNEK1M">Ryujehong’s 70+cm/360</a> would certainly not agree with the WMO!</p>
<h2 id="addendum-stripping-the-mouse-cable">Addendum: Stripping the mouse cable</h2>
<p>The WMO mouse cable really sucks. It’s extremely stiff and doesn’t straighten out easily; this results in snagging and uneven resistance as you move the mouse around, causing the cable to bend and coil. To remedy this, you can strip the outer cable sheath – just (carefully) run a sharp knife around the circumference of the cable, then strip the sheath by gently tugging on it. There is a copper inner sheath, so it should be fine.</p>
<p>You don’t need to strip the entire cable, just a section of it close to the mouse body. I made a cut about an inch from the mouse body, then another half a metre up the cable. I then tugged at the sheath and removed the section. Result: the mouse cable is much more pliable. Can’t really say whether this will affect its durability, but I did this on a £6 second-hander, so I don’t really care too much.</p>
<h2 id="post-amble-i-bought-a-logitech-g403-and-love-it">Post-amble: I bought a Logitech G403 and love it</h2>
<p>I played with the WMO for a week or so; the negative acceleration coupled with my low sensitivity renders it unsuitable. I decided to try something new.</p>
<p>So what to do?. <a href="http://www.rocketjumpninja.com/top-40/">Rocketjumpninja’s website</a> has some great mouse review articles & youtube links, including detailed size comparisons / side-by-side videos of various popular mice.</p>
<p>Given that my hands are fairly long (22cm or so) and I palm grip when gaming, I thought the Zowie EC1-A or the Razer DeathAdder would be good. However, I tried the Zowie EC1-A and it didn’t suit me. It was a good match for me length-wise and was comfortable to rest my hand on, but it’s too wide for me to pick up without changing the way I grip it.</p>
<p>If you have massive hams for hands, definitely check out the EC1-A. The sensor is great, and it has a few other wonderful features, namely a DPI switch being on the base of the mouse (“let’s change DPI to go into sniper mode!” said nobody ever, apart from marketing guys) and zero setup / drivers. True plug ‘n’ play.</p>
<p>After sending it back, I noticed RJN’s videos featured a Logitech G403. The G403 is much larger than the Steelseries Rival 100, and somewhere between the sizes of the EC1-A and the EC2-A. Given that I wanted a larger mouse, but not as big as the EC1-A, the Logitech G403 fitted the bill. It is perfect for my hand size.</p>
<p>Positives:</p>
<ul>
<li>I like the shape even better than the WMO / EC1-A.</li>
<li>It’s light. Not quite as light as the WMO, but within 10g. The extra 10g is not noticeable as…</li>
<li>… it glides much better and faster than any other mouse I’ve tried</li>
<li>The cable slides over a cloth pad easily and doesn’t twist or snag.</li>
<li>The sensor is amazing.</li>
<li>The DPI switch cycles between 400 / 800 / 1600 / 3200 DPI without requiring drivers.</li>
<li>You don’t need to install any software whatsoever unless you want to customise it (I don’t).</li>
</ul>
<p>Negatives:</p>
<ul>
<li>It’s £60, for crying out loud! I managed to get it on sale for £40, but the normal retail price is very steep.</li>
<li>The cable is very thick. While its weight is not an issue, it doesn’t fit in most mouse bungees (razer bungee says: nope).</li>
<li>The DPI switch is on top of the mouse. It’s hard to press accidentally, but it’s still there. Zowie wins this round.</li>
<li>Logitech’s customisation software is a 100+ MB installer. Balls to that! (This is also a major reason I won’t buy razer mice.)</li>
</ul>
<p>I’m pretty sure that my mouse-buying days are over for another 5 years. What I take from this spate of experimentation is that there is no ‘amazing’ mouse that beats all others. It all depends on your hand size, shape and grip style. Try out some mice and see what suits you rather than buying the “best” mouse with the most positive reviews.</p>Mark SimpsonBack in 2011, I wrote a fairly gushing post about the Microsoft WMO 1.1/1.1a. It’s a great mouse. It really is.