What we want to do is a) detect any files which don’t contain tweets, b) remove the redundant user profile, and c) combine the results into a single file.

Well, friends, here is a shell script to do exactly that. You’ll need zsh and xsltproc, both of which are standard on MacOS X and most sane Linuxen.

Output is on stdout, so just redirect to your filename of choice:

$ tweetcombine user_timeline.xml\?count=100\&page=* \
    > tweet_archive.xml

Here’s the script:

#!/bin/zsh

# Combine all of the twitter user_timeline.xml files specified on the command line into a single output
# Written by Alastair Rankine, http://girtby.net
# Licensed as Creative Commons BY-SA

input_args=()
for f in ${(on)*}; do
    [[ -f $f ]] || exit "Not a file: $f"
    input_args+="<input>${f//&/&amp;}</input>"
done

xsltproc - <<EOF
<?xml version="1.0"?>
<!DOCTYPE inputs [
  <!ATTLIST xsl:stylesheet id ID #REQUIRED>
]>
<?xml-stylesheet type="text/xml" href="#style1"?>
<inputs>
  ${input_args}

  <xsl:stylesheet id="style1" version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output type="xml" indent="yes"/>

    <xsl:template match="*">
      <xsl:copy>
        <xsl:copy-of select="@*"/>
        <xsl:apply-templates/>
      </xsl:copy>
    </xsl:template>

    <xsl:template match="statuses">
      <xsl:apply-templates/>
    </xsl:template>

    <xsl:template match="user"/>

    <xsl:template match="xsl:stylesheet"/>

    <xsl:template match="input">
      <xsl:choose>
        <xsl:when test="document(.)/statuses">
          <xsl:apply-templates select="document(.)"/>
        </xsl:when>
        <xsl:otherwise>
          <xsl:message terminate="yes"><xsl:value-of select="."/> does not contain statuses element</xsl:message>
        </xsl:otherwise>
      </xsl:choose>
    </xsl:template>

    <xsl:template match="inputs">
      <statuses type="array">
        <xsl:apply-templates/>
      </statuses>
    </xsl:template>

  </xsl:stylesheet>
</inputs>
EOF

I think this method of sticking filename arguments into an XSL document with an embedded stylesheet is quite a powerful way of processing XML documents with shell scripts. (Probably should put the <input> tags into a separate namespace though…)

But I’m also not happy with the concept of just abandoning the blog, as so many others seem to do. I like the idea of putting it into hibernation, where it still can be linked to and indexed in search engines, but just not active.

So I’ve been working out how to do that. It’s not as easy as I expected. And, yes, worth blogging about…

“Will I Dream, Dave?”

I thought I wanted something quite straightforward. Basically I was going to convert the site into static HTML by walking over it with wget or similar. Then find a host who could serve it up cheaply and reliably.

The cheap part was a requirement but also an expectation. I just thought that someone somewhere would give me some hosting space for static files that would charge about the same as my domain renewal each year. I’m not interested in paying much more than that because, well. I might as well just keep paying the hosting plan I’m on now, and keep the blog open.

I also wanted the new hosting to have cool URIs that don’t change. So if, for example, you’ve bookmarked my insightful not-to-be-missed 2007 post on fixing Ubuntu 7.04 display problems, you’ll be happy to hear that I intended to make sure it serves you the same long-obsolete advice for many decades to come.

And that’s the real trick. As I found out, keeping the URIs the same, is not as easy as it sounds.

wordpress.com

This was the first thing I tried. wordpress.com offers free-to-inexpensive hosting and, hey, I’m already running wordpress, so it should be a snap to switch over. I’d just run the blog as-is with comments disabled and it would be just like in hibernation.

But then I read the fine print, and found that I wasn’t going to be able to keep my current theme with its famous hand-tuned aside formatting. Not ideal, but not a show-stopper either. Also, importing the images and other assets would require involvement from wordpress support. Bit of a pain, but liveable.

So as an experiment I started exporting from my blog and then importing to wordpress.com. How well did this work? Well, about as well as you can expect with software that is not designed to go clunk. Of course, there is no error message to determine what the problem was, nor even an obvious way to erase all the partially-imported content.

Amazon S3

Having resolved by this stage that I was going to convert it to a static site and host it somewhere, I soon came upon the idea of using Amazon’s S3 service. I already had an account, and it looked like a great solution. Cheap, reliable and easy. (Pick any two.)

There are (at least) two problems with using Amazon S3 for this task.

Firstly there is a technical limitation with hosting a “naked” domain (ie girtby.net with no hostname). Basically the way you create a virtual host on S3 is to create a “bucket” with the same name as your host, and then create a CNAME from your domain to that bucketname.s3.amazonaws.com. The problem is that you cannot create a CNAME on the root of a domain, it has to be from a hostname within that domain (eg www.girtby.net will work, girtby.net will not).

So that instantly breaks my URIs, but even if I could solve that problem there’s another limitation. Basically Amazon S3 is not a full hosting environment and doesn’t provide some common web server features. Most notably it won’t serve up / using /index.html.

So I would end up needing my own web server which would perform this redirection. But that defeats the purpose of using Amazon S3; if I had access to a reliable web server, I’d just use that to serve up the site and be done with it.

Google App Engine

Yes, I was quite amazed to discover that you can use Google’s App Engine as a host for an entirely static site. This is an interesting possibility because not only is it a real hosting service, and I can develop software (maybe even a blogging engine!) to bring my site back to life, should I be so inclined.

The plan is almost perfect, but has one flaw. Although Google seem to have once had the ability to host from a naked domain, that ability now seems to have been revoked. Which is a bit unfortunate, as it breaks my URIs.

So that’s really a problem, and one that I must admit I did not forsee when I started using the naked domain as the preferred domain for the blog.

This does seem to be blatant gouging on their part, given that bytes are bytes, and regardless of whether they are destined for a phone or a tethered laptop, the cost is the same. This criticism is warranted in my opinion.

Carriers may claim that tethered laptops inevitably draw more traffic from individual subscribers. But I would suggest that the incremental traffic from a tethered laptop is a lot lower for the iPhone than for other 3G phones. Let’s face it, the iPhone is a pretty capable standalone device, and you’ll rarely need to break out the laptop to get online. Other phones are far inferior at browsing the net directly, and so I’d expect that there is a correspondingly larger proportion of traffic from tethered laptops of subscribers with these phones. This makes the additional pricing seem even more unfair.

But not all of the hate should be directed towards the carriers.

I am yet to see an answer to this question: how do the carriers know which traffic originates from the iPhone itself and which from a tethered laptop? I don’t know the answer definitively but I the iPhone must mark the tethered traffic somehow. I’m guessing that it must pass through the PPP session from the laptop, instead of terminating it in the phone and NATting the traffic.

Regardless, it is Apple that deserves at least some of the blame here for enabling the carriers to detect traffic in the first place. I know of no technical reason why they needed to do this; it sounds like a purely business decision. And one they didn’t need to make; surely the carriers are all Apple’s bitches at this point?

Boo, carriers who charge for tethering. Boo, Apple.

Anyway, one colleague didn’t believe that the answer in the back of the book was correct, and he offered to bet that by running a computer simulation he could prove the book (and me) wrong. I’m not a betting person, but for some reason, possibly euphoria at the prospect of the upcoming partying seminars, I immediately accepted his bet, wagering $100.

What follows is my attempt to win that bet.

So that there is no argument, I’ll reproduce the exact wording of the problem as stated in the puzzle book:

90. Four different pieces of candy are placed in a bag. One is chocolate, one is caramel, and two are licorice. Without looking in the bag, I draw two pieces of candy from it, and place one of them, which is licorice, on a table.

What are the chances that the second piece of candy I have in my hand is the other piece of licorice candy?

My colleague said the answer is ⅓, simply because the candy in the hand can only be one of three other candies still unseen. Of course this is a classic Monty Hall conditional probability problem, and he is quite wrong.

The key insight to this puzzle is that when I (as the person stating the puzzle) am putting the piece of candy on the table I am selecting it. Just as Monty does when he picks the door with the goat. There’s no element of randomness.

So the correct way to assess the probability is to think about the possible combinations of candies in your hand. There are six: the chocolate and caramel, caramel and either licorice, chocolate and either licorice, and the two licorice. Now we know that one of these combinations, the chocolate and caramel, is not possible. There are five remaining possibilities, and one of these is the one we want. Hence the odds are ⅕.

Anyway the agreed method of settling the bet was to write a computer simulation, so I did just that. Here is the output of a sample run:

Out of 1000000 tries, two licorices were extracted 200432 times.
Estimated probability = 0.200432

We have a winner. Thanks JT, cash will be fine.

Here is the C++ code:

#include <stdlib.h>
#include <algorithm>
#include <iostream>
#include <tr1/array>

const long iterations = 1000000;

enum candy {
    chocolate,
    caramel,
    licorice
};

int main(int argc, char *argv[])
{
    ::srand(::time(NULL));

    // Number of times we've pulled out two licorice from the bag
    long two_licorice = 0;

    for(long i = 0; i < iterations;)
    {
        // put the candies in the bag
        std::tr1::array<candy, 4> bag = {{ chocolate, caramel, licorice, licorice }};

        // shuffle them
        std::random_shuffle(bag.begin(), bag.end());

        // pull out two
        std::tr1::array<candy, 2> hand = {{ bag[0], bag[1] }};

        // At least one of the candies we pick out must be a licorice otherwise it doesn't count.
        if (hand[0] != licorice && hand[1] != licorice)
            continue;

        // Count if we've got both licorice
        if (hand[0] == licorice && hand[1] == licorice)
            ++two_licorice;

        ++i;
    }

    std::cout << "Out of " << iterations
              << " tries, two licorices were extracted " << two_licorice << " times.\n"
              << "Estimated probability = "
              << static_cast<double>(two_licorice) / static_cast<double>(iterations)
              << std::endl;

    return 0;
}

Keen observers will have noted that I have tended to blog each time I try out a new version control system, and this really isn’t an exception. Except that, well I’m not just trying it out, I actually use Bazaar daily at $WORK, so and this is like after-hours practice.

Anyway, I wanted to share this because I’ve found that maintaining a staging and production installation of wordpress, complete with custom modifications and a collection of plugins, is a problem ideally solved by a distributed source control. Plus I really like Bazaar, and wanted to show how easy it can be.

I refer to two different machines here, one of which is the production server (ie my hosting provider), and one of which is the staging server (ie my personal machine). Don’t let the fancy terminology put you off; mentally substitute “my box” and “their box” if it helps you.

Setting up a Local Wordpress Instance

On my staging server I branched the wordpress source from launchpad’s Wordpress repository, which is regularly synced with the official repository:

[s] $ bzr branch lp:wordpress

[I'll use an [s] to denote commands run on a staging server, and [p] for commands run on the production server.]

This command creates a "working tree" of the wordpress source code — a set of files and directories — and an accompanying repository of revisions. At any time the working tree corresponds to one of the revisions in the repository, plus any uncommitted changes. Each commit creates a new revision in the repository. Pretty standard stuff really.

So for a new wordpress installation I add the wp-config.php file and commit it:

[s] $ bzr add wp-config.php
[s] $ bzr commit -m "Added config file"

See the codex for other local setup instructions, I just want to focus on the source control tool for now.

Unless you've used distributed version control systems before, you might be a bit wary at this point, perhaps wondering what happens when I next communicate with the upstream repository. But fear not, this is exactly the point of a DVCS. I've created an independent branch, and the parent branch doesn't even need to know about mine. So, I can quite happily make changes of my own and also merge in upstream changes, knowing that it is all tracked correctly.

But for now let's look at going the other way: publishing my changes to the world.

Uploading to the Production Server

One of the cool things about Bazaar is that it supports many different protocols for publishing branches. So for example, I can just push my branch to the hosting server using sftp:

[s] $ bzr push sftp://girtby.net/home/alastair/wordpress

This will create a repository on the remote server containing all the revisions in my local repository. It will not, however, create an associated working tree. Bazaar does not (yet?) support updating a working tree over sftp. I guess there are too many potential issues with local conflicts and such. Anyway the solution is to ssh into the production box and do a checkout of the published branch:

[p] $ cd ~/wordpress
[p] $ bzr co .

What's that? You don't have Bazaar installed on your hosting provider? No problem - all you need is python. Just extract Bazaar into your home directory somewhere, add the bin directory to your path, and you're away. You don't even need to compile anything.

Of course there are many other tasks to set up a production wordpress, but again let's just focus on getting a Bazaar branch associated with the source files.

Another Way

Of course you could bootstrap everything the other way around. Start with a working wordpress installation on your production server, create a bazaar repository for it, then copy that to your local machine. This would be something like:

[p] $ cd ~/wordpress
[p] $ bzr init .
[p] $ bzr add .
[... bzr rm --keep or bzr ignore the files you don't want ...]
[p] $ bzr commit -m "initial checkin"

You would then branch it locally using:

[s] $ bzr branch sftp://server/path/to/wordpress

Staying in Sync

At this point you have two Bazaar branches, and they can easily be kept in sync as follows. First let's make sure we're running the latest wordpress on our staging server:

[s] $ bzr merge lp:wordpress

This just says to merge the latest changes from upstream. As always with merging there is the possibility of a conflict; you make a change that conflicts with the change on the merge source. In general Bazaar is very good at handling these, and anyway you're very unlikely to encounter them unless you're making modifications to the Wordpress core.

At this stage it's a great idea to test the installation locally. Hypothetically, if there were any unit tests, you'd run them at this point. Otherwise, you can just check that the articles display properly, the admin interface works, and whatever else.

You can even see a summary of the changes that you're merging:

[s] $ bzr status -v
modified:
  wp-admin/admin-ajax.php
  wp-admin/custom-header.php
  [... snip ...]
pending merges:
  westi 2009-02-22 Focus on the first blank field when asking for credentials for upgrade/instal...
    ryan 2009-02-22 Allow editing all of a plugin's files. see #6732
    westi 2009-02-22 Wrap the apply_filters call in a function_exists check as this can be calle...
    [... snip ...]

If everything looks OK, commit and push it up to the production server.

[s] $ bzr commit -m "merge from upstream"
[s] $ bzr push

It should have remembered the push location from last time. Also, just like last time the working tree will need to be updated:

[p] $ bzr up

Eject! Eject!

So I've found wordpress trunk to be fairly stable, but should I ever need it, the DVCS provides a safety net.

If I discover a problem after merging in the latest upstream changes, I can quickly revert simply using the entirely surprising:

[p] $ bzr revert

If, on the other hand I only discover the problem after pushing up to the production server, it's still quite easy:

[p] $ bzr uncommit
[p] $ bzr update

I can propagate that change back to my staging server by merging back:

[s] $ bzr merge sftp://girtby.net/home/alastair/wordpress

But like I said, I haven't had to use this.

Synchronising Media

Changes made on the production server are generally easy to sync back to the staging server - I just download a database dump and import it locally. However media such as images are special because they are not stored in the database. Hence you need a way of getting them back to the staging server. I wonder what the answer could be?

Yep, just commit the changes on the production server and merge them back:

[p] $ bzr add assets/2009/01/funny_picture.jpg
[p] $ bzr commit -m "Added funny picture"

[s] $ bzr merge sftp://girtby.net/home/alastair/wordpress

As noted before, the add command can recursively add all files in a directory, so you don't even need to specify the files individually.

Plugins/Themes

Plugins and themes are handled just like any other change. Just extract the plugin to the relevant directory, and add it to the repository:

[s] $ unzip ~/Downloads/coolplugin.zip
[s] $ bzr add coolplugin

At this point I'd probably activate and test the plugin locally, then commit and push to the production server as before.

Bazaar can also track file renames, deletions and moves, but you obviously have to tell it about them. So when coolplugin is updated, be sure to tell Bazaar about any relevant changes before committing:

[s] $ bzr mv --after coolplugin/oldandbroken.php coolplugin/newhotness.php

The --after switch tells Bazaar not to actually do the move, instead it's already happened and we're just recording the fact.

Why Not...

... Wordpress auto-update? Put simply, I don't trust it. Will it let me manage my own patches to wordpress should they be needed? I don't think so. Also: FTP? What decade is this? Even FTPS, sheesh.

... git? Well no reason particularly. I've dabbled with git, but it never really clicked for me. The concepts and terminology and command set still seem slightly obscure to me: "Want to check in your changes? Just use 'git albatross'! Want to view checkin comments from the most recent merge? No problem, 'git ham-sandwich' is at your fingertips!" OK, I exaggerate a little.

... just shut up already? Oh, OK then.

It’s a great little linux distribution that lets you do all sorts of geeky networky things, and is damn reliable to boot. I was a bit nervous about the upgrade from the (now) ancient White Russian release, to the new Kamikaze 8.09_RC2 release, but it all went very smoothly.

If you’ve shied away from OpenWrt in the past because of it’s depencency on command-line installation, well shy no more. OpenWrt now includes an excellent web interface called LuCI, which makes admin very simple. Here it is, showing off real-time network statistics, available via a plugin module.

Modules such as these are the main reason for installing a Linux distro on your router. Different routing modules, statistics and monitoring, security, QoS, application layer proxies, and many more, some with LuCI web interfaces. In short, there’s lots to explore if you’re at all a network geek (I am).

Although OpenWrt was originally written for the infamous Linksys WRT54, but now available for lots of other routers. Perhaps it is supported on the router you’re using right now!? Why not go check?

Nintendo DS Case Swap

A while ago I sat down with some unusual tools and my failing eyesight to rescue a Nintendo DS that had suffered an Act Of 5-year-old. It wasn’t too damaged, but the hinge area had cracked and so it had a tendency to fall apart. At the time I was expecting to pay around $200 for a replacement.

But then I discovered DealExtreme. Oh. My. Dog. That site is so awesome. It’s like a massive south-east Asian computer market but instead of all the cheap cool gadgets being strewn amongst a thousand tiny stalls, they are all catalogued, displayed online and available for purchase with international shipping included. Just amazing, have a browse.

Anyway one of the things you can buy from DX is a full replacement housing for a NDS for $14.15. Sold!

It arrived a couple of weeks later (yes delivery is slow). I spent the afternoon swapping the old case for the new. Put briefly, you do not want to attempt this operation unless you really have to (or have rolled an 18 for dexterity).

The best instructions for Nintendo DS disassembly are available on flickr. I followed them closely and, for the most part, successfully.

Here is a shot I took just after removing the back cover. You can see the broken hinge — which I have completely removed — on the right. After this point things get very tricky. See that ribbon cable on the left? That joins the upper screen to the lower PCB and was a complete bitch to get out.

The new case fit very nicely and everything went quite smoothly during reassembly. The shoulder buttons were only exception to this; it took literally about an hour of fiddling to get them both in and working. In the picture above you can see the right shoulder button still in place, with a little metal hinge that fits into the case itself. Well, when reassembling the NDS you have to align both of the button hinges and both of the little springs (not visible above) that are needed so that when you release the button it doesn’t stay pressed.

In short, I stuffed up one of the springs. So now we have one button that needs to be pushed back out before it can be pressed again. This is livable for most games. Otherwise, everything works perfectly, and has for the last few months. If we want to play a game with a lot of shoulder-button action (eg Mario Kart), we generally play it on the other, pristine, NDS.

Clifford The Cricket

Bouyed by the success of the NDS, the Alien DAC, and by a new-year enthusiasm for making stuff, I bought an electronics kit. The intention was to introduce electronics to the Jr. Girtbys and to re-introduce electronics to me.

This is a relatively simple project that was described in an old (1994) issue of Silicon Chip. The kits are available everywhere, I got mine from Jaycar.

It’s supposed to start chirping like a cricket when you turn out the lights. I don’t really understand how it works, but it’s something to do with a light-dependent resistor which increases voltage across an IC input. This apparently activates a little oscillator and that makes the noise.

Here is mine. It doesn’t work.

I’ve checked, and re-checked, and re-re-checked the components, to ensure that everything is in the right place, and with the right polarity. I’ve also checked my soldering which, while pretty dodgy in places, does at least look to be basically functional.

I can get him to chirp by shorting various pins on the IC – which indicates that it is something to do with the logic parts of the circuit. Perhaps I’ve fried a transistor or something, I don’t know.

Troubleshooting tips appreciated.

Of course I have no real idea what I am doing. Despite having zero knowledge of PHP or JavaScript, for some reason I feel no reluctance towards sitting down at a keyboard and bashing away until I produce either Macbeth or a better website. The entire site is stored in a Bazaar repository, and that makes reverting bad changes especially easy, and I’ll attempt to blog further about that sometime.

For now let me just point out a seemingly small change to the site. The dates of the posts and comments are now displayed in a more human-friendly manner such as “3 days ago” or “20 minutes later”. This was a feature of Typo that I liked enough to port to my Mephisto theme, and which (I thought) would be fairly simple to get into Wordpress.

It wasn’t. Read on for the war story.

The Prototype Version

John Wang describes how it is implemented in Typo. Basically it’s a bit of JavaScript that is run when the page loads. It looks for dates, which are delimited by the abbr tag and have a pre-defined CSS class. The script iterates over each of these and converts them into a friendly date. The abbr tag is handy because it allows you to specify the full date and time in the title attribute, which you can see when you hover over the date with your mouse.

One of the challenges with using JavaScript in this way is to ensure that you register event handlers in a manner that works with other JavaScript on the page, as well as across browsers. The Prototype library helps out immensely here, and for actually locating the abbr tags.

One of the nice features of Wordpress is that it manages dependencies on third-party JavaScript libraries, on behalf of the plugins. So as long as your plugin/theme uses the wp_enqueue_script function, Wordpress will find a copy of the requested library and insert your script correctly into the HTML output. You call it simply from the theme’s functions.php file like this:

add_action( 'wp_print_scripts', 'depo_theme_add_javascript' );

function depo_theme_add_javascript() {
    wp_enqueue_script('friendly_dates', get_bloginfo('stylesheet_directory') .
        '/javascript/friendly_dates.js', array('jquery'));
}

So at this point, I had the friendly date script originally from Typo, registered correctly with Wordpress and, it worked. Sort of. Specifically: when I had some of my Wordpress plugins disabled.

Prototype vs jQuery

Besides Prototype there are also other JavaScript libraries that people use, and they don’t always play nicely together. For example, jQuery is quite popular, and seems to be used by at least two of the wordpress plugins I have installed. Which meant that I could use my date script or the plugins, but not both.

Now jQuery (at least) is supposed to play nice with other libraries if you use the noConflict method. Unfortunately I couldn’t get it to work. I even resorted to asking my first question on Stack Overflow. This seemed to stump the experts too; the best answer was basically “switch to jQuery”.

So that’s what I did.

The jQuery Version

jQuery is a quite unusual library. It reminds me a little bit of XSLT, in that you declare a pattern of matching items in the DOM tree, and for each of those, some actions which affect the output. You might even say the similarity is quite ironic, given that XSLT is the browser’s other scripting language.

By chance I came across yet another pretty date JavaScript from Zach Leatherman (but with ancestry tracing back to jQuery author John Resig). This allowed me to perform the article date transformations almost trivially:

jQuery(document).ready(function() {
    jQuery("abbr.published").humane_dates();
});

Nice. But what about the comment dates? These require two dates, both the comment date but also the date of the article to which it belongs. The former is easy enough, but there was no easy DOM path from the comment to its article meta data. If it was XSLT I could have come up with an XPath expression, but jQuery is unfortunately not that powerful (yet?).

So I settled for a bit of a hack. For each comment, I inserted the article date as a hidden field in the HTML as a sibling to the date abbr, where it can be easily retrieved by a jQuery method.

It’s not an ideal solution of course, and part of me wants to (at least) put the article date in a local cache variable where it doesn’t need to looked up all the time. But another part of me just wants to get it working so that I can start the process of blogging about how I got it working…

Publish and Be Damned

Anyway here’s the code. I’ll find a better place for it if I make any other mods, or if there is any interest in it.

Lessons Learned

I’m not even sure there are many here. Perhaps:

• Don’t mix JavaScript libraries.
• Learn jQuery. It’s powerful.
• C++ is easy compared to diagnosing JavaScript bugs (ok maybe not)

The point is that you, as a mere user of the widget, can’t easily inspect its inner workings to determine the quality of engineering and manufacture, and nor do you typically have the expertise to do so. So you need to rely on the information provided by the vendor. Perhaps it plays on national stereotypes, on technobabble (hello, cosmetics industry!), luxurious packaging (hello, Apple!), or many other techniques.

With software — particularly web-hosted software — it is not so easy. An astute observer will examine the interface of the software in question and look for consistency to established standards, prevalence of modes, and so forth. A lay observer will go on aesthetics, and probably not much more.

As the market for software becomes more and more sophisticated, what are the indicators going to be for quality? “Designed in California“? Or are lay people going to look for some more substantive indicators?

I don’t know but I know what I’d really like it to be. And I know what I’ll be looking more and more closely for, in future.

Made With Unit Testing

Once you’ve gotten the unit testing religion there really is no going back. It fundamentally changes — for the better — the way that you write software.

It can also make you a lot more intolerant towards regression bugs. That’s my excuse anyway for my recent dummy spit over wordpress’ broken comments feed. This is exactly the sort of problem that unit testing can catch. So why wasn’t it? And what assurances do I have that it wont happen again? I am not a wordpress developer, and I don’t care to be one either, but as a user I want to have better confidence in the software I use.

By the way, the fact that I was running an unreleased version of wordpress when I encountered the feed validation bug, is completely beside the point. Regularly run unit tests ensure that even when trunk is broken, it doesn’t remain so for very long. As it turned out, the problem that I encountered did make it into the 2.7 release of wordpress, before being addressed. (Although, as far as I’m concerned, the bug still isn’t fixed, because there’s no unit test to prevent it reoccurring).

Even if you don’t know anything about unit testing, you should be able to factor it into a technology decision. Software package A is unit tested, and B isn’t. Which do you pick, if all else is equal?

Other Types Of Testing?

It might be argued that other forms of testing are an acceptable substitute for unit testing. To which I would say: bullshit. Or perhaps I might say “you are mistaken”, in a more professional forum. Unit tests are by definition automated, and hence a lot more foolproof. Automated testing at higher levels of abstraction is of course possible, but from experience it is far less effective. And manual testing is just not fun.

Alternatively, it might be argued that a more comprehensive indicator of quality throughout the development process is an ISO 9000 accreditation. To which I say: you’ve got to be fucking kidding me. (And let’s defer that discussion to another time).

As an experiment, pick your favourite piece of open source software. Go to the website and work out how to run the unit tests. Check out the latest source code from the repository and see if the unit tests run as advertised. If the unit tests are broken, or (worse) missing, maybe you should start looking elsewhere?

As another experiment, pick your favourite piece of closed source software (if you have one). Ask the vendor whether they have a unit test suite, and if so, what percentage of the code does it cover. Again, depending on the response, you might want to think about switching…

… to software that goes “clunk”.

For a while now I have been in awe of Khoi Vinh’s Subtraction blog, with its amazing and minimal black-and-white design. Since first discovering it I had intended to shamelessly copy it, if ever I decided to redesign my own blog.

The original plan was to follow Matt’s recommendation and customise the Sandbox wordpress theme into some reasonable facsimile of Khoi Vinh’s site, without being too noticeably similar. While casting around for other sites to steal CSS from, I came across a wordpress theme which looked to provide most of what I wanted, so I used that instead of Sandbox. Thanks to Derek Powazek for his DePo Clean theme.

My version of the theme has departed slightly (or perhaps radically, depending on your point of view) from the original. Instead of a 5-column fixed layout, I’ve gone with a mostly-fluid layout. Maybe it’s the coder in me, but it just feels wrong hard-coding magic pixel numbers into the stylesheet…

The fluid layout may turn out to be a mistake, and indeed I’m already concerned about the mismatch between the columns in the footer and those in the body. By the time you read this I may have already changed my mind, and reverted to a fixed-width layout.

One addition I am moderately proud of is the linkroll at the bottom, which is sourced directly from delicious via javascript and styled to match the theme of the site.

For the most part everything has come together nicely but I’m afraid I would be lost without the assistance of the awesome CSSEdit, which as you can see, enables a dilettante designer like me to butcher the creations of others with ease.

I have some further creative additions to the site planned, not to mention some actual content, as well. Happy 2009, folks.