Developer packages are for packagers, not developers

Increasingly over the past few years I’ve become annoyed when building programs from source that my system libraries are out of date. There are several possible solutions:

Give up and wait for one’s distro to be updated with the required version of the libraries (and possibly the program one is trying to build).
Install newer development libraries from an unstable version of one’s distro.
Install newer development libraries from source.

Obviously, the sane option is the last one (assuming that you really do want to build the program from source, and aren’t just impatient to get your hands on the latest version). The middle option, in particular, is fraught with hazards. The only reason one thinks of it is because it’s the obvious, neat thing to do. After all, those library development packages are provided for a reason, right?

Maybe not. After all, development packages can also cause the converse problem: having updated our distro, we discover that a program we have installed from source no longer runs. And then—horror!—it no longer builds with the updated libraries we have.

And there are other problems. Many popular languages have well-developed packaging systems for libraries, such as CPAN and Rubygems, which may or may not be well-integrated into the distribution. For example, in Debian, ruby libraries installed from Debian are not registered as gems, so that if you are trying to build a program which depends on gems, you have to re-install its dependencies as gems.

While development packages are essential for building Debian itself, they should, like source packages, be something that users normally do not install. But development packages are undeniably convenient to install and use, so how do we continue to make it easy to get hold of development libraries? There are two answers to this: for languages with their own packaging systems, use them; for those without, there are already source-friendly packaging systems that can be used. In both cases, distros’ own packaging systems could be more friendly to other packaging systems, acting more as a meta-packaging system. There are many difficulties here; I certainly don’t anticipate ending the practice of natively packaging those libraries and programs needed to build a distro any time soon.

(There is another interesting, wider, discussion here: the idea that a distro should update all its packages in lock-step every six months, from the stable and mission-critical to the barely-used and bleeding-edge, is clearly nonsense. Ubuntu’s 6-monthly updates are a good compromise between stability and up-to-dateness, but breakages in both directions (kernels that no longer support a particular piece of hardware; rapidly developing programs that are out-of-date even before they are shipped) are still common.)

I suggest, therefore, that distros have separate (but probably overlapping) policy for packaging the software needed to build the system from that for installing upstream sources into a development system. In the longer term, this will benefit not just users who are developers, but all users on the one hand, increased choice, as far more apps can be installed using the distro’s standard package manager; on the other, the distro’s developers can stop wasting effort repackaging rarely-used or developer-only software, and spend more effort on that part of the distro that really makes a difference: providing a well-integrated core set of apps, including the ability safely and easily to download and install non-core apps. There’s great scope to use cross-distribution packaging tools like Zero Install; cross-distribution packaging won’t be widely used until it’s integrated with distros’ own packaging tools. Finally, application developers will have much greater incentive to package their own applications, once it can be done in a cross-platform way to which users are likely to have access.

In short, it’s time for package managers to scale up.

Reuben Thomas, 17th December 2010

Last updated 2010/12/17