girtby.net

Some of the criticism of the Blogging Horror article was based on my insistence that knowledge of the C language is essential for all software developers. Some even said I was “bigoted” for such a viewpoint, because there are many other worthy languages out there. And there certainly are. But they are not all created equal.

I had tried to explain my reasons for C’s special place in my list, but maybe I was not clear, so let’s try again.

Here’s the thing: C is everywhere. Recently Tim Bray made basically the same point; all the major operating systems, all the high-level language runtimes, all the databases, and all major productivity applications are written in C. And there are many other categories of software that I haven’t even mentioned, all written in C.

So can you as a developer choose to ignore it? Live in high-level language land for your entire career? I would say almost certainly not. High-level languages often provide abstractions that relieve you from the burden of dealing with the platform on which you’re building. Which is great, but sooner or later a crack is going to open up and the abstraction is going to leak.

Some day you’ll need to go spelunking into the depths of your runtime environment. Maybe you’ll need to call some other C-based API for which you don’t have a convenient wrapper in your high-level language of choice. Like, say, mmap-ing a part of a file instead of the whole thing. Or maybe you’ll just want a bit of a performance boost. And on that day, boy will you wish you knew C.

It’s for this pragmatic and entirely non-bigoted reason that I promoted C to the top of my language pantheon. If you’ve never learned C, it means you’ll never be able to delve too deeply into the foundations your programming environment and find out exactly what is happening under the surface, or to extend it in any way.

Of course there are many other reasons to learn C, such as those discussed in comments previously, but this is main reason why it’s on my essentials list.

Unfortunately the Rudd government are pressing forward with their proposal for mandatory internet filtering. Recently, Electronic Frontiers Australia summarised the results of an analysis of current ISP-level filters commissioned by my old mates at ACMA. The figures are frankly begging to be plugged into Bayes’ Theorem, so let’s do that.

Continue reading...

These days, everyone’s a photographer. There can be no doubt that the advent of the digital camera has provided vast numbers with the means to explore their creativity through photography. As such there is a vast and growing industry just to support the great unwashed in their quest to take better photographs, or at least to advise what camera to buy.

So there is absolutely no need for anyone else to be dispensing photographic advice. But here I go anyway!

My unique angle, such as it is, is that I recently had an … unscheduled equipment upgrade. Otherwise known as a burglary. This meant starting afresh with all-new shiny toys, courtesy of the insurance company.

This is the first of a series of articles in which I gloat review some photographic kit and provide some hopefully helpful advice along the way.

But firstly let me disclaim that I have no photographic credentials to speak of. You probably know more about photography than I do. But then again maybe you don’t. Maybe you can view my shots (below) and decide for yourself.

Continue reading...

<movie-trailer-guy> Many months ago he attempted Tim Bray’s first Wide Finder in C++, mainly as a coding exercise. Back then the goal was readability and conciseness. This time … it’s performanceal. </movie-trailer-guy>

Continue reading...

Let’s solve Twitter’s scalability problems, shall we?

So, like most people, I don’t know much about the problems there and certainly don’t have any solutions to suggest. But I do know there are a certain class of solutions which aren’t on the table.

If you look at Twitter from a suitably high vantage point you see real-time communication between small groups. People entering short messages and having these messages appear at their peers a small time later. There’s also a central archive, but I’ve heard Twitter described as “public Instant-Messaging” and this seems to characterise it best for me.

In short, Twitter seems more suited to peer-to-peer communication than to client-server. What sort of protocol would it use? I can imagine a protocol which would be probably UDP-based, and which would send tweets to followers either directly from peers or perhaps through a local aggregation point. Large groups of followers could perhaps even use UDP multicast. Archive servers could be reached through network anycast addresses, to allow for greater decentralisation. IPv6 to get universal connectivity. And so on; fill in your own pet network technology here, there are certainly lots of potential solutions.

Instead of these, clients communicate directly with the Twitter servers using HTTP. Not only that, but they poll for updates. Bit of an architectural blunder, you might think. Well not really. In fact I don’t think the Twitter designers had any choice.

Once upon a time it was possible to deploy new application-layer protocols on the Internet. But those times have passed, it seems. These days, it’s HTTP(S) or nothing. And this is not the protocol you would choose for carrying tweets, if you had the choice. So the fact that twitter works at all over this sub-optimal application-layer protocol is quite an achievement.

This is a great example of the many ways in which innovation can be stifled by enforcing a lowest-common-denominator.

The impact is of course more widespread than just Twitter. In fact, the so-called end-to-end principle which was one of the fundamental founding principles of the Internet is now all but abandoned in practice. Geoff Huston examines the issue in some detail in a recent article, and it is highly recommended.

Of course, there are no easy answers, either for Twitter or the next application to suffer due to the proliferation of network middleware. But it’s certainly an issue that does need to be more prominent.

(This post is an obvious departure from my usual style of blatant attack pieces in order to score traffic and fame for myself. Normal service will resume shortly.)

So you’re watching a new TV show and you’re hooked. It’s clever, the characters are believable, the dialog is witty, the cinematography is inspired, the direction is tight and the plot is engaging. You want to see more. You’re in love. With a TV show.

Continue reading...

As you know, I’m a fan of the DRM free music. In fact it seems that I’ve blogged about it each time I’ve discovered a new website that sells the stuff. And the latest discovery is emusic. They have hits and some misses.

Continue reading...

A long, long time ago in a corporate universe far, far away… I was admitted to an elite group. A group where the members each had a manifesto. Chris was a founding member of the group, and has since published his manifesto on his blog. I don’t quite know how I was deemed worthy for membership. My manifesto was unpublished, it was mostly unfinished, and it was unseen by all but me. Regardless, I was admitted. A fraud. Living a lie. For years I have lived with this shame.

At last all can be revealed, for here is my manifesto. Or as I originally called it, my List of Skills and Knowledge That All Developers Should Have. There are 9 items in no particular order, so I guess that makes it a set rather than a list for all you pedants in the audience (and I know you’re there).

The idea is to document a set of skills that every developer should have. That’s everyone who develops software professionally. Doesn’t matter what type of position or company or industry; this is my stab at a body of knowledge that every serious developer should have. Essentials, essentially.

It’s partly based on experience; specifically the experience of surprise I felt when a colleage, or random stranger on the internet, expressed ignorance at one of the items on this list. Other items on the list have come from an exacting process of posterior extraction. I’ll leave it to you to guess which is which, and who is who. Or at least skim the headings. Read on.

Continue reading...

This weekend, Peter and I built this:

It’s an Alien DAC; a USB to analogue audio converter that has a Burr-Brown PCM2702 chip at its heart.

I’ve been listening to it a bit today and am very impressed with the quality. In fact, to my ears it sounds even better than the DAC built into my Corda Move amplifier. Upon plugging it in I immediately noticed some musical details in the treble that I hadn’t noticed before. Of course, I haven’t double-blinded myself, so maybe I’m imagining it. Regardless, the Alien is certainly hard to fault. And from only $50 of parts.

I got the parts as a kit from Glass Jar Audio; recommended. I hate to think what a painstaking job it would be to collect so many parts from various suppliers for these type of kits.

Construction credit goes to Peter; I mostly just watched and made light conversation, with the exception of capacitor C16, with is mine dammit!

The surface-mount components are very difficult to solder. My hands just shake too much! (Got to cut down on the coffee apparently) With the pins on the PCM2702 itself, Peter had to solder them all together in one big blob and then soak up the excess solder using a wick. The increasing use of surface-mount components is apparently causing a bit of controversy in the homebrew-electronics community.

I must disclose that it didn’t quite work first time, but we found the problems by a fairly simple process of following the circuit diagram and checking the voltage with a multimeter at key points along the way (we didn’t get to C16 thankfully). In the end the problems were relatively simple cases of short-circuiting and easily fixed. I’m pretty certain this technique goes a long way in electronic kit troubleshooting.

It was great to watch someone who knows what they are doing, and I am inspired to build some more electronic components myself (though perhaps not surface-mount).

Over to the right — unless you are using one o’ them fancypants RSS aggree-gators — are the monthly archives for girtby.net, with a count of the number of articles posted for each month. Last month there was just the one and this month isn’t looking too promising either. That’s as good a measure as any that I’ve been, well, preoccupied.

It’s a pretty safe bet that if you see a semi-regular blogger (still not quite ready to label myself like that, but anyway) suddenly go quiet, then they have either had a baby or a new job. And so it is with great pleasure that I can announce that I am the proud parent of … a 5- and a 7-year-old, and also I have a new job!

I like it so far, there’s a lot of c++ coding about which I intend to blog copiously until every last remnant of a regular audience has fled for their lives. Whilst simultaneously, and endlessly, reciting the latest internet meme to jump the shark (eg “FAIL!”). Oh yes.

But not to worry because I promise not to drive you all away just yet - not until I have at returned from a 2 week northern hemisphere (just) vacation. Tomorrow, the family and I are on the next plane out of Sydney, and after seven flying hours we’ll be landing in Phuket. More interestingly we’re also venturing inland to a rather remote part of the country, staying in a villa about which you can read on their excellent website. In short, I’m very much looking forward to it.

Naturally I have some reading matter and some new toys to sustain me for the flight, and I will make a vague, non-committal promise to blog about one of these on my return.

Keep the internet warm for me while I’m gone, won’t you?

Specialised web forums are commonplace these days. They cover the entirety of the long tail, and are therefore indispensable for discussing obscure, and not-so-obscure, topics.

These days I wouldn’t consider making any serious purchase without at least a brief consulation of the relevant web forum. Technical problems with just about anything can usually be resolved with a well-crafted search through the appropriate forum. Put simply, the web forum is the basic unit of online community these days.

But despite their ubiquity and obvious utility they remain frustratingly limited in lots of ways. In this post I vent some of these frustrations.

Continue reading...

Those of you familiar with my propensity to succumb to temptation for the latest and greatest may have been looking at my setup, marvelling at the mildly dated hardware, rolling your eyes and thinking “that’s not going to last long”. And I’m happy to confirm that, in accordance with prophesy, the hardware upgrade juggernaut has finally rolled through my part of the world. Oh my word yes.

In the first week of January, Apple announced an upgrade to the Mac Pro line of workstations. They would have 8 cores as “standard”, through the use of two 4-core Harpertown Xeon CPUs. What they didn’t highlight was the fact that the “standard” configuration was actually customisable down to a mere single 2.8GHz 4-core CPU, making it available for a comparatively-quite-reasonable A$3300.

And naturally, I bought one. Read on for initial impressions.

Continue reading...

A few years ago I purchased an Arc-AAA led flashlight, mainly on the advice of Dan. It was an excellent piece of kit. Small enough to carry everywhere but bright enough to be useful. For years it was the only thing that hung off my keychain (besides keys of course). A month ago it died due to a battery leak.

I wasn’t hopeful of saving it. Since I bought my unit, the guy had stopped selling them for a while, and then been bought out by another company. Nevertheless I wrote to the new company and asked if it could be fixed. They said sure, send it in. A few weeks later, this arrived:

A brand new Arc-AAA, mysteriously labeled an Arc-P.

The new one is slightly heavier I think, but feels more rugged. The knurled metal case is that dull greenish-grey colour, as might be carried by Brown from Spook Country. The light is activated by twisting the head, and the new unit has a lot more resistance which makes it a lot less likely to turn on in your pocket. It’s at least as bright as the old Arc-AAA, in other words surprisingly good for the size (and capable of out-shining any AAA Maglite). If you want to get really mental I see there’s now a “premium” version which is even brighter.

The quality of the unit itself and of the support from the vendor gets two thumbs up from me.

Yesterday’s apology to the Stolen Generation was a very moving and hugely significant moment in Australian history. For most of my life the many problems of Aboriginal people in this country have seemed intractable, even hopeless. This is the first time in many, many years that visible and meaningful progress has been made. I hope the importance of the occasion is adequately reflected in the global news coverage.

The speech itself (video) deserves special mention I think. It pleases me greatly that Kevin Rudd found just the right words to represent my view and (I hope) that of most other right-thinking Australians. Worth a listen.

Local news coverage has of course been blanket, and peppered with the obligatory contrasting views.

The most famous of the contrary views is that of the ex-PM John Howard who notoriously claimed that he saw no reason to apologise for past deeds because they were committed by a previous generation. This has a simple, possibly obvious rebuttal, but I have not heard it mentioned in the media so I’ll state it here.

If (hypothetical) you wish to renounce the wrongdoings of the past, you are free to do so. However, to be consistent you must also renounce the achievements of the past.

Can you imagine John Howard renouncing the sacrifices and achievements of the ANZACs? His past sporting heroes? His hero Robert Menzies? No, I can’t either. However it is necessary to do so in order to avoid the responsibility for the Stolen Generation.

In simple terms, you have to take the good with the bad.

For Aboriginal Australia, the good has been in short supply of late, and it’s very gratifying to see signs of change.

I generally agree unequivocally with Bruce Schneier, but his recent column on vendor lock-in has me wanting to take issue with some of his points.

Vendor lock-in is real, but the example he gives of the iPhone is not a very good one. Why? Because it’s easy to switch: you call up the carrier (AT&T in this case) and say “I don’t like my iPhone, it’s too sleek and good looking and it’s user interface is too elegant. Instead I’d like to subject myself to some nonsense from the traditional handset vendors.” To which the AT&T person says “sure, we’ll charge you $X and ship out a new handset. When it arrives, just activate and transfer your contacts.” Bingo, you’re off the iPhone.

[Update: Andrew points out in comments that the 24-month contract may impede switching in this manner. I don’t know the details, but I’d be surprised if it was impossible to switch away from the iPhone, merely expensive. This is, to my mind anyway, not sufficient to justify the term “vendor lock-in”, but I suppose that depends on your definition. My definition is below.]

In Australia we have number portability which means that I can generally switch handset or carrier without too much fuss. I’m not sure about the situation in the US, but as illustrated above you are still free to switch handsets while keeping the same carrier. So if there’s lock-in at play here, it’s lock-in to AT&T, not the iPhone.

So what is vendor lock-in anyway? I would define it as the presence of constraints on a given product or service that are imposed by the vendor and which prevent you from switching to a different product. These constraints may take the form of missing features which would enable a switch, or of usage constraints imposed by licensing, or both. Either way there has been an explicit decision — technical or policy — by the vendor which prevents switching to a competitive product. Hence the term is a mild pejorative.

It’s a slightly confusing term because it applies to a product or service, and not to the vendor. So it’s quite possible for product X to exhibit vendor lock-in, but not product Y from the same vendor. “Vendor-imposed lock-in” might be a better term.

Note that there is an implicit assumption that the features and capabilities of the product in question are available elsewhere in the marketplace. In other words, there exists an equivalent product to switch to. This assumption does not always hold, and sometimes you may find yourself unable to switch to a different product, simply because there are no other products on the market with a given capability or feature. This does not, by my definition anyway, constitute vendor lock-in, because the inability to switch does not arise as a result of a decision from the vendor.

Does the lack of an SDK constitute vendor lock-in for the iPhone, as claimed by Schneier? Well, does the lack of this feature prevent switching to a different product? No, of course it doesn’t, as illustrated above.

In fact, it is the presence of an SDK which constitutes vendor lock-in, of a sort. Third-party applications written to the iPhone cannot, by definition, be easily be ported to other mobile platforms. If you suddenly decide you don’t like your iPhone any more, but have hundreds of third-party applications installed, you have a problem.

This problem is common to all computing platforms; vendor lock-in is a necessary consequence of all vendor-controlled SDKs and APIs.

Incidentally the delay in making an iPhone SDK available can quite easily be explained by the technical challenges involved, and does not neccesarily imply any policy decision by Apple to deliberately lock out third-party developers. Producing an SDK of any quality is a hard task, and the instant it is released it has to be supported for the life of the product. As Charles Miller puts it, “third party apps are for life, not just for Christmas”. It is quite understandable that Apple would make sure their SDK is just right before committing to it.

But where does the “no SDK == lock-in” idea come from anyway? I suspect that it arises from the expectation that we are able to install third-party applications on the iPhone. Where does the expectation come from? It comes from the disclosed fact that the iPhone runs OS X. If Apple had not divulged this fact, or if the iPhone ran some un-named OS — as is the case for all classic iPods, for example — there would be no expectation of third-party applications. It is for this reason no one is claiming that the lack of an iPod SDK exhibits vendor lock-in.

However, Schneier claims that there is* vendor lock-in on the iPod, due to the fact that “music purchased from Apple for your iPod won’t work on other brands of music players”. This is misleading; it is quite possible to purchase DRM-free music from Apple for the iPod and other players. Again, he’s incorrectly identified the source of the vendor lock-in, which in this case is *certain music from the iTunes Store and not the iPod.

To reiterate, vendor lock-in is real and is important. It is contrary to the idea of Free Data and deserves to be more widely discussed. However, let’s first understand what we are talking about, so that we can think critically.