Killing applications, not killer applications

One of the problems with commercial software is that it still largely sets the benchmarks for what all software developers want to produce. Hence, apart from a few mavericks most developers are following the established GUI metaphor, which encourages (though it is not by any means originally responsible for) the tendency to think in terms of "applications". This suits companies wishing to sell software, because applications can be arbitrarily large and complex, and have arbitrarily many features added; they are never complete. Even the modern versions of the classic UNIX command-line "tool chest" programs, which are often cited as examples of "software tools" designed to perform tasks in combination rather than as individual applications, suffer from the application mentality: grep and sed, for example, include their own subtly different regular expression engines.

What should we be doing instead? Simply, making programs work better together. Some of the problems are not even with the "application" mentality, but because programs are often hard to combine flexibly. For example, grep needs extra code and options in order to use Perl-compatible regular expressions, and either a wrapper script or yet more code in order automatically to decompress compressed files. "Libraries" usually require code to bind to them, and in the second case although you can achieve something close to the desired result by "preloading" a compression library that overrides the standard file-reading routines, the approach lacks fine control (e.g. what happens when you really do want to grep through a compressed file) and opens potential security holes (by overriding system libraries).

There are undoubtedly technical problems to be solved: programs have a wealth of ways to interact (pipes, sockets, temporary files, forking, dynamic linking, scripting) and they need fewer and more flexible ways, but this brief rant does not attempt to address this problem.

Instead, I want to return to the problem of the "killer application". The UNIX tool-chest may not be ideal, but it achieves much more with much less (code, programmer time, computing resources and hence user expenditure on hardware and software, and arguably user frustration) than bloated, bug-ridden (because large, complex and hard-to-test) killer apps. Much can already be achieved without any new technical solutions.

I suddenly realise that I'm at a loss to give examples, as most of my work in this area has been in the less dramatic and more subtle region of programs that are already lightweight, whether it is simplifying and removing bloat from a console editor, working on grep (sadly by adding code) to simplify its use and (even more remote from users) its distribution as part of complete systems, or working on a sound converter to reduce its internal complexity and allow it to support new sound formats without writing code (by making its various readers, writers and filters into standalone programs, rather than, as at present, reimplementing the pipe system inside SoX). Most of my simplification over the last ten years has consisted not of programming, but simply of choosing programs more amenable to toolkit, compositional use: rather than Windows Explorer or its free software equivalents, I use the bash shell, which for all its shortcomings (and they are legion) remains one of the most comfortable readily-available ways to interact with a computer; for editing I use Emacs, arguably a classic piece of bloatware, but also an environment for building editing applications that has attracted thousands of programmers to add routines for performing particular tasks, so that it works remarkably closely with a wide range of other software. I use only six applications daily: editor, shell, email client, web browser and two messaging programs, and that is five too many (one application is the same as zero), but I haven't yet managed to combine them further. It is the desire to do so that started to make me think about this sort of reductive programming.

Well, that was a ramble. Most of the value was probably in the headline. I suspect there are some tangled ideas that need untangling. Comments, as usual, welcome.

Reuben Thomas, Battersea, 14th November 2006

Last updated 2006/11/14