Log

Vala Vala: Why I’m giving up on Vala for now

[This post was written for vala.dev but not published there.]

Hi! I’m Reuben, a long-time free software maintainer and author. One of my specialities is taking on mature projects from their original authors and maintaining them long-term. (This is the same introduction I wrote last year in my previous blog post; it’s all still true!)

Having now worked with Vala on several medium-sized projects (500-5,000 lines of code) projects over the last 5 years, I’ve decided to stop using it for new projects for now, because the pain points are outweighing the benefits for me. I thought it worth addressing them in a blog post; in another timeline, this is my to-do list for improving Vala!

In my previous post, I looked at two projects I have translated from C to Vala: the minimal Emacs clone GNU Zile and the spell-checking meta-library Enchant, which has been used by GNOME for years, most recently by libspelling, which optionally provides spell-checking in the GNOME Text Editor.

Before continuing, I want to say that I had a great deal of fun with Vala, I am sad to be putting it to one side for now, and I’d certainly consider using it again if there were signs that the shortcomings I’ve identified were being systematically addressed. Further, I’m sure that for many uses it’s still a good choice, and in any case I’ll continue to maintain the projects I have released. In particular, for Enchant, I don’t regret using Vala at all, and I still have plans to develop the project further, using the facilities that Vala provides more thoroughly to make Enchant easier to understand and maintain.

My last straw

Recently, I translated another, smaller code base (about 500 lines of code) into Vala: the command-line text search-and-replace tool rpl, which was previously written in Python. Although I mostly use rpl on small files (typically, documents in plain text formats) where the performance of the Python version is entirely adequate, some users run it on multi-gigabyte or even larger inputs, and were finding problems. I wanted to help these users, and had problems with both speed and memory consumption that I found hard to solve in Python. (I did not try using the Python C API, or extensions such as Cython. If I were starting the project today, I might do this instead.)

This foregrounded some problems I’ve run into in the past, and in this project they were harder to avoid. I’ll go through them below, but first…

Why I’m not fixing Vala

Rather than changing my tools (with all the costs that involves), couldn’t I just fix Vala? I have in the past made quite a few contributions in my areas of expertise (in particular POSIX and GNU bindings, and to the build system), but I don’t want to get involved in the compiler. Others have already tried to fix some of the problems I’ve run into, in some cases, some years ago. Sadly, in the last couple of years development effort on Vala itself have all but stalled, so the bar to contributing to the compiler has risen from “convince a maintainer to take my patches” to “become a maintainer”. I have even some fairly simple and isolated fixes to VAPIs still waiting to be merged.

And some of the problems that were most serious for me do indeed need fixes to the internals compiler, unfortunately, though many could be solved without such drastic intervention.

For me, there are sufficiently good alternatives, whether sticking with C or C++, using Python and its ability to work with C when needed, or learning Go, whose more mature ecosystem makes up for the greater effort it requires to interface to C. (I have prototyped a version of rpl in Go, and may switch to it one day.)

In short, in Python and Go I have to do a bit more work to talk to C, but I don’t have to fix the language or standard libraries, and C bindings I might want often already exist.

The types of problem

The problems I ran into fall into two main categories:

Language problems. These could be fixed with a new profile, i.e. without breaking backwards compatibility.
API problems. These are problems inherited from GLib. Fixing them in GLib is probably a whole other discussion, but just as libgee provides better containers for Vala than GLib, a Vala-specific library could fix the problems with the I/O APIs of GLib.

Let’s dig into the problems.

Array indices are `int` by default

Array sizes should be size_t. This has been known for a long time (issue from 2012). Recently, Vala 0.56 added support for specifying the type of an array’s length. But it is not the default, and many APIs still use int. This makes it hard to write code that safely deals with arbitrary amounts of data.

Binding problems

There are problems in the Vala bindings for core APIs such as GLib. Some of these can be fixed (indeed, I have contributed several fixes over the last few years). Some can be worked around by using existing alternatives, but the broken versions are still a stumbling block to developers.

Stat is broken, as you can’t tell when it failed. Workaround: use Posix.Stat.
size_t is not actually size_t, but gsize. This can cause C compilation failure when a Vala size_t is passed to a C API expecting size_t. Similarly for ssize_t.

GLib-induced problems

GLib is a comprehensive and capable library that is beset with paper-cut problems in its APIs. Its I/O abstractions are a mess: there are multiple overlapping abstractions, each of which have certain functionality and lack others. My current application, rpl, unfortunately, is quite demanding in some of the areas where GLib is weakest.

Here are some examples, along with suggested solutions. The suggestions are all Vala-specific. Of course, GLib itself could be improved, and is, gradually over time. But moving further and/or faster would presumably take a great deal of effort, and might well not be a priority for its core users, which I presume to be authors of Gtk applications. However, better Vala bindings and libraries based on GLib can offer benefits to Vala developers much more easily, as well as pointing to improvements that could be adopted by GLib itself.

GLib assumes that strings are NUL-terminated

This means that when dealing with strings that may contain NULs, one cannot use the most convenient forms of APIs (in particular, Vala’s nice way of binding GLib so that you don’t need to pass an explicit length). Some APIs do not have a version that allows an explicit length to be passed.

This could be fixed with a new string type in Vala. It could be source-compatible with the existing type, replacing the current length accessor with a concrete property, and using that property as the default value of length arguments in methods.

`int` is used where `size_t` should be used

Or at least ssize_t. For example, the Regex APIs use int (gint in C) for character offsets, even though the length of the string is size_t (gssize in C).

This example and others cannot be directly fixed in Vala, as it’s a problem with the underlying GLib API. In the case of Regex, it would be fairly straightforward to duplicate the API and reimplement it with correctly-sized indexes. One could take the opportunity to expose features of PCRE2 that are currently not available, such as different character widths, and the features required for streaming operation. (For rpl, I wrote my own PCRE2 binding, since I needed functionality not exposed by the GLib API.)

File APIs are a mess

Want to do file I/O? Great, FileStream has you covered. Oh, so does InputStream/OutputStream. And IOChannel! None of these is deprecated, and they all have different functionality. InputStream/OutputStream can be easily sub-classed and composed (this is the API I would like to see recommended in Vala). But IOChannel is the only way in Vala to read text a character at a time.

Some classes are unnecessarily non-portable. For example, UnixInputStream and UnixOutputStream, which are used to wrap file descriptors as InputStreams and OutputStreams respectively, could work on Windows, but don’t, as they are in the gio-unix package, which is not built for Windows.

For Vala use, the easiest solution would be to write a new I/O library, along the lines of libgee, which one could view as providing better container types than the GLib-based ones. Something similar to the InputStream/OutputStream model, which hews closely to the Java and Go standard library APIs, both of which lack the rough edges of GLib, would be both straightforward to write (copy the mature existing models), look familiar to Vala and GLib programmers, and be easy to extend.

One example: rpl needs to use some of its input twice: first, it needs to use some of it to detect its character encoding, and then it needs to process the entire stream. I achieve this by copying the data that is used to detect the character encoding, and then feeding it to a custom subclass of InputStream, called PrefixInputStream, which combines a block of bytes with an InputStream, and acts like an InputStream that concatenates the two. (The Go standard library already has a way to compose streams, just saying!)

Character conversion is badly designed

Vala gets access to “raw” iconv, which is great, but apparently not much used, since I found and fixed a bug in the VAPI binding.

However, as usual, the code is nicer if you can use InputStream and OutputStream classes. These exist: ConverterInputStream and ConverterOutputStream. But there’s a problem: ConverterInputStream can return partial characters. Since it’s an InputStream, it returns a number of bytes, not a number of characters. Fair enough! But it will happily return part of, say, a UTF-8 code-point, meaning that when reading from it, one has to re-validate the input if one expects to work on valid UTF-8. A similar problem occurs with ConverterOutputStream: it will write partial characters if the destination is not ready to receive the whole array.

It’s worth nothing that IOChannel fixes this, but only for UTF-8 input, and further its structure means that it can’t be straightforwardly sub-classed in Vala.

Building and using large strings is inconvenient

GLib, and hence Vala, offer the StringBuilder class to deal with making large strings. It keeps the string in a single array, and doubles the size of the array every time it needs to be reallocated, which amortizes the time cost of allocation. However:

There’s no way to directly append bytes from an InputStream into a StringBuilder.
A StringBuilder cannot be used as an OutputStream.
There’s no way to manually reallocate a StringBuilder’s memory, which would be helpful when you know exactly how much is needed.
The GLib function g_string_set_size for safely setting the length of a StringBuilder is missing from the Vala binding. Ideally the setter for length should do a bounds check and write the NUL.

Vitality problem

Vala has been developed slowly for the last 5 years, but at least to start with it was possible to upstream fixes and some reported bugs were fixed. Now it seems pretty much to have ground to a halt. That’s an additional disincentive to trying to fix these problems.

Summary: what would get me to reconsider?

A new “Vala 1.0” profile

Implement and release a new profile that fixes the fundamental type problems. It should also include improved default VAPIs: move the legacy GLib APIs into a compatibility module, foreground newer and updated APIs, and add the new I/O library I mentioned (if necessary; it might be possible to write all or at least most of it as a VAPI).

Make it the future default, reassuring users that the current problems are not forever, and encouraging users to migrate their code. The use of profiles means that old code would not be abandoned, just eventually it would have to use an explicit --profile vala0 switch, or similar.

Updated documentation

I only discovered the ability to specify the type of array lengths while writing this article. I have submitted a patch to the Vala tutorial, but while doing so I noticed that the old language specification has not yet been transferred to the vala.dev web site, and hence there’s no way to update it, marooned on the old Wiki. Vala needs an up-to-date reference specification!

Updated packaging

Relatedly, it wasn’t until I worked on rpl that I even found out about vala-extra-vapis, which is unfortunately not packaged e.g. in Debian. By that time, I had already duplicated one of its bindings.

More developer time

This is a Catch-22, of course: if Vala had more developer time, these problems might already have been fixed. At least, with the velocity of a few years ago, existing patches would be being upstreamed and release, and there would be a feeling of progress.

I’m not that interested in working on improving Vala: I’d rather be using it to write code. I’ve spent time on Vala in the past because of its advantages over other languages. Increasingly, other languages are overtaking it for my purposes, so the incentive for me to contribute time to Vala improvements is reduced.

This is a shame, because Vala is a great language, and I’ve invested considerable time and effort learning how to use it productively. Perhaps most frustratingly, the effort required to fix the problems I’ve outlined has mostly already been made: it’s largely a question of “getting it across the line”. The changes I’ve suggested above are within the scope of a single stable release cycle.

Conclusion: “fare well”, not “goodbye”!

It may be that I’ve overlooked even more recent improvements like the ability to specify array length types, and that actually all my pain points are already cured. If not, I shall be exploring other, non-Vala avenues, but I’ll be casting occasional glances “over my shoulder” to see how Vala gets on. I hope it does get the attention it deserves to fix some of these problems at last, and make it once again more attractive to programmers who have or want to work in a C-based ecosystem but yearn for a way to do that that’s more pleasant and modern than C, and less rebarbative and perplexing than C++.

>The Big Conversation

>Hay 2008

Vala: the smoothest C off-ramp

[This is a post originally written for vala.dev]

Hi! I’m Reuben, a long-time free software maintainer and author. One of my specialities is taking on mature projects from their original authors and maintaining them long-term.

In this post, I want to talk about how I’ve rewritten entire projects in Vala, to make them easier, more productive and more fun to maintain.

Vala works well with C—that’s one of its primary features and design choices. But these abilities can be used for another purpose: to provide a “smooth off-ramp” for C, converting old code step-by-step to more-maintainable Vala, without an “all or nothing” rewrite, and with the ability to reimplement C libraries in Vala, so that existing C code can still call them.

Over the few years, I have rewritten two small–to–medium-sized C code bases in Vala: the minimal Emacs clone GNU Zile and the spell-checking meta-library Enchant, which has been used by GNOME for years, most recently by libspelling, which provides spell-checking in the GNOME Text Editor. In the rest of this post, I’ll focus on the challenges I faced doing these rewrites, how Vala helped me, and discuss whether such a drastic move (of rewriting entire code bases) was in fact a good idea.

Vala is good at talking to C

Vala is designed to work well with C, and a nice consequence of this is that often one doesn’t have to think about C at all, because so many useful libraries are already conveniently bound to Vala “out of the box”. It’s impressive the degree to which some of Vala’s basic abilities, such as its “built-in” container types are “just” clever binding of GLib functions plus some syntactic sugar.

But Vala’s binding abilities are very powerful: it’s not just GLib and GObject APIs that it can bind successfully; it can bind almost any C API. Later I will also show some examples that test its limits.

Further, Vala can implement C APIs just as well as it can consume them.

Allowing C to talk back

Why would you want to implement C APIs in Vala? I have three answers for you. Firstly, maintaining a typical C code-base is a horrible experience. I just wanted to not have more memory management errors, especially to do with strings (which both Zile and Enchant are big on). It’s so easy to corrupt memory, to get lifetimes wrong, or simply get the wrong answer with C’s primitive string manipulation routines.

Secondly, implementing an existing C API in Vala gives an easier-to-maintain codebase that can be used by existing consumers, which don’t need to be C programs: they can be written in Python, Rust, JavaScript or, of course, Vala!

Thirdly, and perhaps less obviously, you can use this ability to rewrite a C program in Vala incrementally, one module (say) at a time. The result is a pure Vala application (no C APIs are being implemented), but at each step one has a complete working application, partly written in C and partly in Vala; and where C needs to call Vala, the Vala code must implement the right C APIs.

Before we start, a sanity check

I’m talking about rewriting mature code-bases. Zile dates from the late 1990s, and I rewrote it into Vala in 2020. Enchant was written starting around 2003, and I rewrote it in Vala this week (in 2024). So:

Isn’t that an awful lot of work?
Won’t it introduce lots of new bugs?

In other words, is it worth it? Or am I just being quixotic?

It’s not that hard

As I said above, I rewrote Enchant in Vala this week. Actually, that’s not quite true: I rewrote the core, about 2,000 lines of C. Enchant is, as I said above, a meta-library: that is, it doesn’t do spell-checking itself, but delegates to a number of libraries that do. Each one needs an adaptor plugin, which Enchant calls a “provider”, and these providers are written in C, C++ and even Objective C. I haven’t touched those (yet!).

But still, 2,000 lines of code in a week is not bad.

GNU Zile was about 8,000 lines of C, and that took me about a month.

Avoiding a plague of new bugs

Vala is well matched to C

Any time you touch code you can introduce bugs, of course: the slightest adjustment can be risky. How much more so rewriting in another language?

The good news here is that because Vala is so close to C, it is relatively easy to translate code without introducing new bugs. The syntax is sufficiently similar that code can be easily translated with some search and replace (e.g. NULL in C becomes null in Vala) plus light editing (e.g. adding missing comparisons in conditions to make them boolean, so that if (ptr) in C becomes if (ptr != null) in Vala). C APIs can be used directly via Vala bindings, and C data structures can be replicated precisely. Vala’s static typing also helps.

Here’s an example of a struct in C from Zile:

struct Region
{
#define FIELD(ty, name) ty name;
#include "region.h"
#undef FIELD
};

#define FIELD(ty, field)                  \
  GETTER (Region, region, ty, field)      \
  SETTER (Region, region, ty, field)

#include "region.h"
#undef FIELD

The file region.h contained:

FIELD(size_t, start)
FIELD(size_t, end)

Here’s the translation into Vala:

public class Region {
    public size_t start { get; set; }
    public size_t end { get; set; }
    // …methods go here
}

Compare and contrast: around 2009, I translated Zile into Lua. I completed the translation, but I did not release it, as I found that Lua’s lack of static type-checking made it too easy to introduce new bugs. While I was translating it I was forever finding off-by-one errors caused by Lua counting arrays and string positions from 1, while C is 0-based (as is Vala, of course). Don’t get me wrong: Lua is a great language; indeed, I have successfully written other, smaller programs from C into Lua, though I wouldn’t now recommend that; its strengths lie elsewhere.

With Vala, I recommend making the initial translation as “light” as possible: don’t use lots of Vala’s features to rewrite code. Translate char * pointers to string offsets (that is, an integer string index). Make sure the translated code works first, then start simplifying. Even the “light” translation will contain simplifications, anyway: no more malloc or free, for example.

You already have a test suite, of course

The other way you avoid a plague of new bugs is with a good test suite. Sorry! There’s no way around that. If you want to alter, let alone completely rewrite code, you need to have tests.

Fortunately, I had already added a fairly extensive test suite to Zile years ago, when I was trying to make sure that it really behaved exactly like Emacs. The situation with Enchant was even better: it has an exhaustive (and sometimes exhausting) test suite that checks every aspect of its API, added a few years after Enchant was written, by a commission from SIL International. (A big shout-out to those who commission work on libre software, especially this sort of important under-the-hood work. These tests have saved me from introducing bugs time and time again.)

Again, you have to be disciplined: no turning off the tests, or changing them as you go. Complete the translation without changing them, as far as possible.

Your existing build tool is probably supported

Many Vala projects these days are using the Meson build system, which is very popular in the GNOME space. But Vala is also supported well by older tools, such as CMake and GNU Autotools. Both of the projects I translated were using GNU Autotools, and I stuck with it, for two main reasons:

I use the excellent gnulib portability library to ensure that my POSIX-compliant code runs on most systems without problems. Unfortunately, gnulib is tied to GNU Autotools. But then, few build systems work on as many platforms as the Autotools in any case.
Automake not only has Vala support, but allows you to ship the generated C sources, so that users can build your code without a Vala compiler. This means that your translated project has exactly the same build dependencies as it had before, which can help with users and packagers who haven’t heard of Vala and don’t want to learn. With Zile, previously a pure-POSIX project, I added GLib as a build-time dependency, but of course that’s so widely used that it’s unlikely to be a problem. Enchant already used GLib.

The Vala experience

I will now describe the process of translating the two code-bases. Although they were a bit different (Zile an application, Enchant a library), and I made the translations a few years apart, I was pleasantly surprised to find the second time that the process was much the same as the first.

Add Vala to the build system

With Autotools, this is a matter of adding a couple of configure.ac lines to detect the Vala compiler and check for GNU Make (which is required to build from Vala sources), and then adding a test Vala source file to a Makefile.am.

Translating the code

I typically translate one file at a time. The code will of course usually call functions from other parts of the program: I add a private VAPI file with suitable bindings. You can usefully pre-populate the VAPI file with the commented-out contents of relevant C header files, then just uncomment the APIs you need in Vala, and edit them suitably. The excellent Manual bindings guide from the Wiki is my constant companion.

To translate a file, I copy foo.c to foo_vala.vala. (The _vala bit ensures that the Vala-generated C won’t overwrite the original file, which I will keep for reference.) I translate the code, possibly in multiple steps, commenting out the bits I have translated in foo.c.

Compile, test, fix, rinse, repeat.

I find that I get a nice rhythm with my translation, of doing the obvious basic syntactic stuff, as mentioned above, then finding the equivalent Vala APIs where necessary, then tackling any hard bits, such as binding third-party APIs that Vala does not yet know about.

For Zile I leaned heavily on the POSIX VAPI, and added a new GNU VAPI for some GNU-specific APIs, which I have also used and extended while translating Enchant. Enchant was easier on the whole, because it was already GLib-based, so I just had to search valadoc.org for the Vala versions of the GLib APIs.

For Zile, I found libgee invaluable, with its superior collections that are both richer than GLib’s and allows extra use cases, such as collections of unboxed simple types (e.g. ArrayList<int>). For Enchant, it so happened I didn’t need this extra functionality; Enchant’s use of collections is mostly limited to singly-linked lists of objects and hash tables with string keys.

Stumbling blocks…

Documentation and API coverage

Vala is a small ecosystem, even after all these years. Hence, I found bugs, mostly in the documentation. I took the time to submit fixes, and this past week reaped the rewards! Some of these fixes went as far as GLib itself; many were to the Vala wiki. As I noted above, I also made additions to the VAPIs, in particular adding a complete binding for the POSIX, GNU and BSD regex APIs (Zile only needed the GNU API, but since the GNU regex module implements all three, I thought I might as well!), as well as getopt_long(). I added freopen to the POSIX VAPI (as FILE.reopen()), upgraded the curses VAPI to improve its Terminfo coverage and add key-codes for function keys, and added return codes to GLib’s FileStream.putc and puts.

I was also able to offer my old-school expertise to the Vala project itself: I converted the test suite to use Automake’s relatively recent ability to run tests in parallel.

Falling between two stools

Vala itself mostly insulates you from the memory-management woes that bedevil C, but mid-translation there is still C code, so you have to be careful if you want to keep a working program. It’s always tempting to just plough on with the translation and hope that crashes will disappear, but I try not to give in, as that can bury bugs deeper and make them harder to find later. So, expect to spend some time studying valac-generated C, and revising your knowledge of owned and other esoteric Vala features.

For me, this is particularly relevant to Enchant, as a C library whose API and ABI I want to maintain. I am happy to report that I have been mostly successful, and at the time of writing I am tracking down some problems with object lifetime management, but otherwise the Vala implementation seems to perfectly replicate the C.

Another area of awkwardness is strings: as I hinted above, C often uses pointers to refer to string positions, whereas in Vala it’s usual to use integer offsets. This code is fairly easy to convert once you get the hang of it. Since C can use offsets instead of pointers, you can always rewrite the code in two stages, first changing the C to use offsets, and then translating it into Vala.

Here’s an example of the translation from pointers in C to offsets in Vala. The C version refers to positions in strings with pointers:

char * enchant_iso_639_from_tag (const char * const dict_tag) {
    char * new_tag = strdup (dict_tag);
    if (new_tag == NULL)
        return NULL;
    char * needle = strchr (new_tag, '_');

    if (needle != NULL)
        *needle = '\0';

    return new_tag;
}

In Vala, we use string offsets:

static string iso_639_from_tag(string dict_tag) {
    return dict_tag.substring(0, dict_tag.index_of_char('_'));
}

The comparison here is perhaps obscured by the simplification obtained in Vala from automatic memory management, which is nothing to do with string handling per se.

Binding challenges

I had a number of particular challenges. Some of them may be relevant to other developers, but I detail them here mostly to illustrate that Vala can bind just about anything if you try!

`config.h`

Like any Autoconf-based build system, mine produce a config.h header of various #defined symbols, some of which my C code uses. These I simply replicate in a Config module

One could imagine generating this automatically from config.h, but since I didn’t need more than a handful of values in my projects, I didn’t bother.

`relocate`

This is a GNU API that returns a string that may or may not be malloced. You can tell which easily enough: if the return value is different from the argument, then it is a newly-allocated string. The problem is getting Vala to do the right thing. Even in C, I wrote my own wrapper function that would always return a malloced string.

Here’s the Vala VAPI definition:

[CCode (cheader_filename = "relocatable.h", cname = "relocate")]
private unowned string _gnulib_relocate(string path);

I use the name _gnulib_relocate because I am about to add a Vala wrapper which uses the original name. We tell Vala that the return value is unowned, to prevent Vala thinking that it must free it.

Here’s the wrapper, which I also put in the VAPI:

[CCode (cname = "_vala_relocate")]
public string relocate(string path) {
    unowned string newpath = _gnulib_relocate(path);
    // If relocate malloced, then return the value, defeating Vala's attempt
    // to (re-)strdup it.
    GLib.return_val_if_fail(newpath.data == path.data, newpath);
    // Otherwise, allow Vala to strdup the non-malloced return value.
    return newpath;
}

This is tricky. First, it calls the “real” relocate (but using its Vala name), and stores the result in an unowned string. So far, valac will not allocate any memory.

We then test the value returned, and if it is unequal to the original argument, return it. Because we use the return_if_fail macro, valac will not insert memory allocation code at this point, but that’s OK, because relocate has already malloced in this case, so the return value is an allocated string as indicated.

Otherwise, we simply return the un-malloced value. In the generated C, valac will insert a g_strdup at this point, so again the return value is an allocated string.

Phew!

`strfreev`

Rather surprisingly, while g_strfreev takes char ** (because it frees a list of strings), strfreev takes string **. Why? Well, Vala uses one level of pointer indirection to indicate that a value is not managed by Vala. So we have the following set of equivalences:

string = char * (generates char *; as we expect, a string is a pointer to char)
string * = char * (generates char *; this is just a string that Vala does not manage)
string ** = char ** (generates char **; now we have a list of string)

This was a problem for me because I wanted to use strfreev to free a “real” char ** passed in from C. But valac gives a type error. I got sufficiently confused that I submitted a merge request, and Rico had to de-confuse me. In the end, all I needed was a cast:

strfreev((string **)c_string_list);

The cast placates valac, and the generated C ends up calling g_strfreev, which expected a char ** all along.

`[s]size_t` is not really `[s]size_t`

When compiling my Vala version of Enchant on Windows, I accidentally tried compiling it in a flavour of Windows I don’t normally test, namely Mingw MSYS (I normally build for Mingw64-x86_64 and Mingw64-i686). To my surprise, I got a compilation error: GCC complained that one 8-byte signed integer type (long int) was incompatible with another 8-byte signed integer type (long long int). Yes, these are both 64-bit types on this system, and yes, they are nonetheless incompatible.

But why did I have this problem in the first place? I was using Vala’s ssize_t, and Vala maps this to GLib’s gsize. I don’t know why, but it’s a problem, because in some cases, like mine, these two types end up with different fundamental types in the C compiler, and an error results. I filed an issue and found a workaround, which was to copy the definition of size_t and ssize_t from posix.vapi, which are normally only used when valac is used with --profile posix and the GLib definitions are not used. The posix.vapi definitions correctly map size_t in Vala to size_t in C.

…and helping hands

I mentioned above that Vala is a small ecosystem. An up-side to this is that it’s friendly and usually quick to get things done. The fixes I offered were mostly triaged and applied by the indefatigable Rico Tzschichholz, who was often able to suggest more elegant solutions, mostly for my VAPI bindings. I ended up using a pre-release of valac for Zile, but I could release the code, since you didn’t need valac to build it, and within a few months, Vala 0.52 was released with all the fixes I needed.

With Enchant I have had help from Rico again, and also from Lorenz Wildberg; many thanks to both of them, and to everyone else in the community to has both helped me with my projects, and welcomed my own contributions, which, I hope, improve things for everyone else.

Was it worth it?

With GNU Zile I can say: probably, yes. I found one “brown-paper bag” bug in the initial stable release, but that’s it: I have made only two non-alpha releases of the Vala version. On the other hand, the C version was already very stable, and I suspect it’s not widely used any more (I myself have encouraged users to migrate to Emacs for some years!). But then, it was a low-risk project to try out this new technique. And a few years on, the code still builds with no problems: unlike some language ecosystems, Vala seems very stable, and code does not rot fast.

For Enchant, it’s too early to say. It’s a much more widely-used package, and though it is quite mature, I have steadily improved and developed it over the past few years, and intend to continue in future. I hope that the switch to Vala will not annoy packagers (it shouldn’t, as they shouldn’t have to change their recipes at all if they don’t want to), and that developers should not notice (because the library remains API and ABI compatible), nor users (because the functionality should be identical, and the performance much the same). Certainly, I expect it will make my future work more productive and fun!

Is this a good idea?

My projects are old-school: highly portable, low-level C. This is not an area that Vala is particularly aimed at, although it does pretty well here. For projects that are firmly in the GLib world, and especially when they use related libraries such as Gtk, rewriting C in Vala seems an excellent option to me.

While I completely rewrote my projects before releasing them, I think I’ve shown there’s no need to do that. When I rewrote Zile in Lua, the intermediate steps had horrible performance, because the Lua/C interface I used was very inefficient (Lua has an efficient-but-complex C API; I chose to use some simple-but-slow home-grown macros instead). When I rewrote Zile in Vala, the performance was much the same at every stage of the process, and I could easily have translated just half of the code and released the project. Many projects, particularly larger ones, might benefit from such an approach.

After completing the translation, the code was already significantly shorter and more readable than in C. This in turn unlocked further simplifications: able to “see the wood for the trees” it was much easier to refactor and further simplify the code. With Zile, I eventually reduced 8,000 lines of C to 6,000 of Vala, a 25% reduction; in Enchant the percentage saving was more like 35%, largely because of a reduction in GLib boilerplate, such as argument validation, which Vala mostly automates.

One final thought: I hope that Vala will continue to flourish for many years, but if it doesn’t, then all is not lost: in the worst case, I or my successor could salvage the generated C code, which is ugly but far from impossible to read (and C, I think we can confidently say, will be around for many decades yet); and Vala is also a relatively “neutral” language that would be easy to translate again, into Rust, say; indeed, it would be easier to do that starting from Vala than from C, largely because the Vala code is easier to read and understand.

You are always invited to join our matrix chat #vala:gnome.org and ask questions about Vala or how to contribute. Until then, have a nice time and build great apps with Vala!

Eats shoots and leaves

Lynn Truss’s guide to punctuation is unappetising fare, even for pandas.

Eats, Shoots & Leaves purports to be a witty guide to the correct use of English punctuation which also strikes a blow for ‘sticklers’ against the prevailing culture of ignorance and indifference. Sadly, what could have been a pithy and amusing Strunk & White of punctuation is swollen by its author’s dotty schoolmarm act into a 200-page rant that exemplifies the vices it affects to make light of, and displays woeful ignorance and stupidity in every field but that of punctuation, not to mention much unpleasant prejudice.

Suffused with the pedant’s impression in every age that standards are in decline, fond memories of her “manual typewriter in the 1970s” (page 135) contrasting with today’s “bloody electronic signal” (page 190, followed a few pages later by an evisceration of (text) emoticons); she yet admits that things have never been perfect: on page 39, she quotes the Oxford Companion to English Literature: “There never was a golden age in which the rules…were…known, understood and followed by most educated people”. This have-and-eat approach to cake pervades the book.

Even where she sticks strictly to her subject, she is often unauthoritative: her seventh use of the apostrophe (page 45) is to indicate the plural of letters, but she fails to mention that this practice is not universal, as I was taught at school, and rediscovered when translating a book for Cambridge University Press. She is also inconsistent, on page 50 listing the mail-order firm Lands’ End as an offender in their punctuation of their name (under “Plural possessive instead of singular possessive”), but on page 57 excusing St Thomas’ Hospital on the reasonable grounds that “institutions, towns, colleges, families, companies and brands have authority over their own spelling and punctuation”.

Her bad taste ranges from the lazily stereotypical (“greengrocers are self-evidently horny-thumbed people who do not live by words”) through the casually bigoted (page 16, “Around this time, when other girls of my age were attending the Isle of Wight Festival and having abortions…”; describing dialect on page 44 as “strange, non-standard English” and on page 39 referring to “the regrettable result of making people sound…a bit from the West Country”), to the offensive: on page 186 she quotes Bernard Shaw’s letter to The Times in 1945 complaining about the waste caused by not spelling ‘bomb’ without the final ‘b’ and says “GBS can be a pretty stark reminder of how far one may lose one’s sense of proportion when obsessed by matters of language.” She has obviously forgotten that on pages 4–5 she wrote “[sticklers] got very worked up after 9/11 not because of Osama bin-Laden but because people on the radio kept saying ‘enormity’ when they meant ‘magnitude’”—a distinction without a difference, in any case; ‘enormity’ can mean ‘magnitude’.

Using Kingsley Amis’s taxonomy (page 30), she’s an intellectual berk and a syntactic wanker.

One more thing: in Truss’s version (back cover), the panda “fires two shots in the air”. That smacks of bowdlerization, and it’s not funny. In the version I heard, he massacred everyone in the restaurant; it seems that Truss had a lucky escape.

18th August 2017

Child-like

What is special about children? Why are we encouraged to be like children or stay in touch with our inner child? Often, it’s the ability to retain a sense of wonder at the mundane, and this is important. Sometimes, children’s playfulness is mentioned. But what is crucial about playfulness is the ability to take things seriously, things that are not on the list of socially-approved Serious Things (God, mores, money).

6th November 2014 / 17th May 2017

An unreadable book on usability

Sadly, Ben Shneiderman’s book Leonardo’s Laptop, rather than taking its inspiration from da Vinci and using his ideas to inform an intelligent foray into usability, conscripts the great man as the author’s mouthpiece, and contrives to spew almost uninterrupted marketese, in the process getting under the skin of the thoughtful reader by mis-quoting Star Trek, glibly asserting that computers will never be creative, and presenting as the ideal future a vision that could have come straight from a Silicon Valley press release.

Fortunately, Schneiderman does have one significant point of difference with the techno-utopian mainstream, and that is his insistence that technology must support human goals and be adapted to them rather than the other way around. He cleverly attacks the idea that technology should aim to replace humans by turning it on its head: ”The goal of making computers do what humans do, the replacement theory, also seems rather modest. Imagine proposing to make a bulldozer that lifts as much as the strongest human, or a printer that writes as fast as the best human scribe.” (p. 238), and while many readers will groan at the inane rhetoric of ”Why can’t every student earn an A?” (p. 112), he succeeds in making those who would replace humans look misguided and unimaginative. He does not, however, address the issue of technologies that modify, rather than merely amplify, human capabilities, and his soft-focus futurism (”Imagine that after a sunrise climb you reach the summit. You open up your phonecam and send a panoramic view to your grandparents, parents and friends. They hear the sound of birds, smell of mountain air, feel the coolness of wind, and experience your feeling of success.” (p. 2)) suggests that it’s something he’d rather not think about.

Schneiderman is an HCI specialist, and Leonardo’s Laptop takes the obvious approach for such a practitioner: after an introduction that uses Leonardo to emphasise the importance of technology that supports human values, he shows us a glimpse of how the future could and should be, diagnoses what is wrong with current computer interfaces, connects this to our low expectations and misguidedly technocratic approach to technology development, sketches the HCI approach to designing computer systems, and develops a systematic framework for it, applies the ideas to the individual consumer’s view of education, retailing, medicine and government, and ends with a discussion of creativity and a final chapter that returns to the theme of “promoting human values” (About this book, p. 14).

In the course of the book, Schneiderman develops some useful tools for thinking about how people use computers. His Activities and Relationships Table (ART, pp. 87ff.) is a simple two-dimensional matrix associating four stages of creative activity (Collect, Relate, Create, Donate) and four levels of relationship (Self, Family and friends, Colleagues and neighbors, Citizens and markets), allowing technologies to be placed on a map by the activities they support. His last chapter, “Mega-Creativity”, offers a simple and usable list of types of creative task (p. 219). These tools are unsophisticated and obvious, but Schneiderman’s willingness to dive into the tricky area of creativity and offer a straightforward discussion is refreshing. His discussion of situational creativity (dealing with the importance of one’s environment and relationships) was particularly interesting, as it’s frequently neglected in other discussions. This attempt at a comprehensive framework may underpin the admirable breadth of Schneiderman’s application areas, from the personal to the political, including health, education and commerce.

Unfortunately, some of his examples are ill-chosen. The idea that the excellent Sim City “teaches deep lessons about urban planning” (p. 222) is news even to this high-school student of human geography, as is the description of Dramatica Pro, a tool for writing film scripts based on a theory of story-telling, is “compelling” (pp. 222–3). (I would have been much more impressed by a discussion of Inform, or other interactive fiction authoring systems.) At the end of the book, when discussing what use a modern-day Leonardo (“Leonardo II”) might make of technology, he gives the example of using email and instant messaging for collaboration. This is not stirring stuff for a book published in 2002.

These examples are also typical of the software Schneiderman both describes and, more tellingly, envisages: large closed-system proprietary solutions such as LEON (a collaborative online learning tool, p. 116) and, more nebulously, the World Wide Med (information for medical professionals, p. 175). He has a standard capitalist attitude to copyright, yet cheerfully uses public domain prints of works by Leonardo. His two-page discussion of freedom in a political context (pp. 184–5) does not mention free software. Again, curious given when the book was written.

I should confess that Schneiderman lost my sympathy early on with his p. 52 attack on “obscure” text-based interfaces, rightly criticizing the élitist culture of some devotees, but ignoring their advantages, particularly in the internet age: text is easy to store, search and transmit (between computers and humans). Graphical user interfaces also suffer from obscurity, and are often less amenable to user modification. It’s all the odder given that he later lauds text-based chat (p. 199), and that text-based interfaces can much more easily be adapted to level-structured learning (p. 47).

However, Schneiderman’s biggest weakness (and, one is tempted to think, his closest point of contact with Leonardo), alluded to at the start of this review, is his unwillingness to address the political and social context. Despite castigating an opponent for arguing that an issue ”was outside his concern and something for policymakers to decide” he rarely does more than state the political and social obstacles between him and his goals, and say that research or public debate will be necessary. In his summary chapter on ”Mega-Creativity”, he blandly ends ”Participatory design methods and comprehensible, widely disseminated social impact statements may be effective because they promote discussion and expand the range of options for decision makers.” (p. 231) This is barely more than a restatement of the problem; and a tendency to hedge (”support for innovation could lead to positive changes to our world” (p. 230)) gives what should be his boldest passages the flavour of a corporate press release that had to be approved by the company lawyers. At a smaller scale, he has little idea what to do with misbehaving individuals: his discussion of education does not mention discipline, or, in the context of collaboration, ethical behaviour by students, whether in the sense of contributing fairly, or, that bug-bear of educators in the internet age, plagiarism.

His human-centred focus also leads to an unfortunate dismissal of AI. “Even serious scientists are prone to considering artificial consciousness as a good, useful and attainable research goal.” (p. 236) There is indeed a debate about whether artificial consciousness is “good, useful and attainable”, but it is odd that Schneiderman sees no human-centred applications for it, given for instance Japanese efforts to make artificial companions.

On the other hand, Schneiderman also repeatedly skates over potential down-sides, as with his ”World-Wide Med” system for globally shared medial records, or ”LEON” framework to support collaborative learning: his enthusiastic description of imaginary future successes is not balanced by his brief ”Skeptic’s Corner” sections, which do little more than acknowledge that there are problems and say that they are worthy of solution.

The book ends: “Those who believe that they can shape the future will shape the future.” Unfortunately, the author does not seem to be a believer.

15th May 2017

Bookshore: a library is a liminal thing

In French, “librarie” means bookshop, a usage that seems prophetic now that so many small (and not-so-small) Amazon booksellers’ stock consists of what would once have been their owners’ libraries. The books are become interchangeable, still fulfilling their function as extra insulation, but with their old secondary function of entertainment and enlightenment replaced by that of enrichment. (I imagine that some particularly cleverly-balanced people combine reading with commerce, but not many; the Amazon stock at Thomas Towers is pre-packed for the post and stored in the attic, as much to protect the rest of the collection from abduction as to grease the wheels of commerce.)

I’ve spent the last couple of days overhauling my browser bookmarks, of which I have some 1,300 (coincidentally, when I catalogued the family book collection as a teenager, it amounted to roughly that number of volumes). I filed a hundred or so amassed over the past four or five years, and combed through 300 identified as broken links, updating them where possible, in some cases pointing them at the astonishing Wayback Machine (at least an important a public service as Wikipedia), and in a few cases lamenting their utter disappearance.

But the sort of link I’m interested in right now are the sort I simply deleted: links I no longer needed. Later, I realised that I’ve started treating my browser bookmarks the way I treat my book collection: as a boundary, a shoreline.

The hinterland is the contents of my brain: what I’ve learned, can recall, can rely on. A few mines of information are represented on my shelves, and in my bookmarks, by reference works: dictionaries (actually, I mostly have these on my phone now, as vadite mecum and for speed of use) and other essential works; online equivalents are mostly esoteric, as the ones I use all the time need no bookmark (Google, Wikipedia &c.).

The ocean is everything I do not know, will never know, have no wish to know.

And the shoreline: that is where the books and bookmarks are littered. Everything I am interested in, the pebbles of knowledge, the seaweed of YouTube & thrillers, the curious creatures of mine and others’ unknown selves. Here, my physical library is rather skewed, as thanks to the unconscionable length of modern copyright combined with the irresistible collection left by my maternal grandparents, it consists mostly of books I have or would like to read once, but must keep in case, as the former would be too expensive to replace, and the latter to acquire. At least the classics are safely removed to the still-expanding Project Gutenberg, which should surely replace the Bible-and-a-Shakespeare as standard issue on Desert Island Discs.

But the rest is largely a matter of memory: if I might forget it, I bookmark it. If I don’t know it, I have the book. If I’m simply not interested, why give it window or wall space? Disk space is another thing altogether: my computers are full of data I will probably never need, but, unlike with books, the probability can be effectively zero, as that is the chance I’ll ever fill my disk with text. That’s not the same as bookmarking everything, as bookmarks take effort to navigate, and hence occupy brain space.

So I suggest that, where of old a library might have been merely indicative of its owner’s interests and knowledge, much still gained from other sources, or simply on subjects not treated by the published word; or, more recently, a solid image of its owner, books representing reading, actual or aspirational; now, it’s more likely to constitute an shoreline, knowledge behind and ignorance ahead, and hopefully the tide is going out!

Wed, 15 Mar 2017 00:10:03

Restauration

I am in a restaurant. The owner and chef has gone out for dinner. I am sitting at a camping table, on a comfortable steel-framed chair such as you might find in a modern conference hall, in the corner room of a down-at-heel château dating from 1892. I am alone in the house. The window in front of me is open to a cool cricket-filled August evening, a strip of lighter sky still visible on the tree-lined horizon below ominous clouds promising rain but failing to deliver. Above the chirping I can hear the whine of a nearby water treatment plant. It is nearly half-past ten. The only light in the room comes from a shaving lamp which hangs askew over the basin that serves for washing both oneself and up. The one other working light in the house is next door in a disused bathroom. To my left hums a fridge, and on a table next to it are a kettle, a toaster, an electric toothbrush, an improvised draining board covered with plates, cutlery and a saucepan, and a five-way adaptor that comprises the sole usable source of electricity. The walls behind and to the right of me are stacked with boxes, most unopened, some half-unpacked, almost hiding two more tables.

Beside my laptop, which in the dim room has the signal virtue of supplying its own light, is a copy of the Ordo Solesmensis 2012/13. The abbey of Solesmes is just down the road onto which the property gives at three points, only one of which is passable; another is blocked by a gate marked “PROPRIETE PRIVEE / DEFENSE D’ENTRER” a few yards short of the road, and the last by a hoarding on which is visible a sign whose most prominent lettering says, on the side facing the house, “A VENDRE”, and on the side facing the road, “VENDU”. The first leads to the walk-in dialysis centre which occupies the lower ground floor; the new owner directs visitors and employees to park in its reserved spaces at the back of the house with plastic hoarding that blocks the drive in front. Inside the corner door which is at present the château’s only functional entrance is another sign, “PECHE INTERDITE”, soon to be erected on the path to the pond below the house, and intended to deter fishers rather than fructophiles or such perversely licentious persons as may haunt the environs of monasteries.

Earlier today, the fire brigade visited, not, thank goodness, owing to any incendiary threat, but because a large wasps’ nest was discovered on the second floor; a hazard which, I know from previous visits to the area, sometimes afflicts stately homes of the Sarthe. In France, such infestations are of public concern. The main hall, currently shuttered, was littered with the corpses of vanquished wasps when I arrived earlier this evening, and still faintly buzzing with their more tenacious brethren.

The château, disused for several years, was most recently an old people’s home. Surgical bed frames are still to be found, and communal toilet and bathing facilities lend an institutional air. Alterations made to accommodate this generation of residents obscure the original logic of the space: one may pass a fireplace in a narrow hall, or open double doors to discover a wall. Some such walls have rude holes poked in them by the new owner’s irritation at these architectural insults; one he has already demolished.

Cables festooning the hallways in a curious range of sizes and colours prefigure a brighter future as I find my way to my bedroom by torchlight. It has two sinks (“use the right-hand one”) and a mattress neatly made up in the middle of the newly-exposed wooden floor, with a small bedside chest of drawers next to it. Many of the rooms are still covered in plywood vandalically nailed directly into the underlying planks and faux-cosily carpeted. On the low-ceilinged second floor, bedrooms of oddly different size cluster under the eaves, some still with Dymoed names on the doors. A small tower room up a couple of steps looks like it would make a promising nook. One of the towers is occupied by a lift, needless to say out of order. The one still-operational feature of the ancien régime is the series of illuminated exit signs, each dim “SORTIE” glowing somewhat weaker than the beams of the compact but powerful torch with which I’ve been issued.

Usually I would feel uneasy left alone in a large secluded building, but for some reason I don’t here, perhaps because I’ve not yet seen it tenanted. It is solid and quiet too, with none of the discomfiting creaks one would expect in a smaller timber-framed house.

All in all, the château seems an almost ideal place for my purposes: to spend a few relaxing summer days, including, I hope, some hours finishing off a bit of poetry I have in hand, and perhaps starting some more. The only obvious defect is the lack of hot running water, but my friend the owner assures me that his rented flat with a shower is only a few minutes away by car.

But what is he doing here? A professional musician, whose occasional talk in the past few years of opening a restaurant I never took seriously until he gave me the videophone tour of the château he was about to buy, the property having at the last minute edged out of his affections a local restaurant which had come up for sale and attracted his attention in the first place, his enthusiasm and talent for cooking had never suggested a future career to my cautious and rule-bound way of thinking. While I have happily sat up until 3am waiting for his tarte au citron (the best I have yet tasted), and there are few phrases that make my mouth water so readily as his self-deprecating announcement, when I visit, that he will feed me “just something simple”, there are a number of roads still to travel from the cottage of such companionable fare to the market-town of a sustaining business. A few minutes down the road from the abbey where he was once organist for a year, the room in which he intends to open his restaurant currently houses his organ. Picking me up from the nearby TGV station in a smart suit and clapped-out borrowed car, he explained that he was to dine with a group of local supporters of the catering trade, with a bias towards the local organic produce which he first delightedly described when he informed me a few months ago that he was selling up to move to this little-known region of west central France. (It later turned out this was a discreet cover for “I have a first date”.) From my place in the ultimate coach, number 20, I walked all the way back down the platform to the station exit, somewhere around coach 5, almost as far as I had walked up the platform at the Gare Montparnasse in Paris, the obvious implication that the gradus ad Parnassum only go half-way to the real destination. Having stationed his banger akimbo in a spot reserved for taxis, my dear friend was just entering the modest station’s hall from one side as I came in at the other, beaming and gesticulating as he chatted on the phone to a friend back in Paris, extolling the rapidly-developing delights of his new establishment, and inviting him down for a barbecue, which in the absence of a cooker is as much necessity as luxury.

After a whistle-stop tour of the house, I presented Pimms, lemonade and the compliments of some mutual Parisian friends, we drank a glass of wine together, and he headed off for dinner, first to find an acquaintance and restaurateur from the northern part of the département who was supposed to pick him up, but had lost her way. After nibbling some sustaining bread and cheese, I idled my way around the grounds in the interval before sunset, choosing not to try the semi-decayed bridge over the lily-strewn pond, which, although its iron frame looked insufficiently rusted to break under me, I preferred to avoid without at least someone else present. In the roadside bay where the “VENDU” sign informed passers-by that the place was recently sold, two cars were parked, of which one was my friend’s shabby loan with its wheelbarrow in the boot; sure enough back at the house, as I crested the main stairs to catch the view, my friend and his acquaintance had just come around the opposite corner. As it was already after nine, I surmised that his typical insouciance towards arrangements must have triumphed over a normal dinner hour (for the Spanish may dine after ten, but we are still a long way north of Spain).

I waved them off and returned to the makeshift kitchen; found gazpacho, melon, more bread, and finally settled down with coffee, dark chocolate both “MILD” and “KRÄFTIG” which, whether because the top-most squares had been exposed for some time, or because of my wonted impatience, I could not tell apart, and the half-dried apricots that seem for some years now to have driven out the dried sort: delicious but messy to eat, and irritatingly sticky in muesli.

One of the defects of an apparently orderly mind, I realised a while ago, is that it likes to impose itself on other people; indeed, I say “apparently” precisely because in laziness it tends to fasten on others’ ocular motes in preference to its own harder-to-shift beams. But this is not my script to write; I have not even the editorial responsibility that a parent or teacher may feel, and so I am free to marvel at the scale of the challenges to be overcome, and to imagine the most pleasing future I can.

And thus I dream that I am writing this in the nook at the top of the tower. Up the stairs wafts the clatter of the last of the dessert plates being cleared and the chatter of the staff as they tidy up and of the final guests as they leave. A few, staying in the hotel which the top floors became a couple of years after the restaurant had extended its sway over the entire upper ground floor, trudge upstairs. From outside, I can just catch singing accompanied by a guitar; though the gastronomic guests have displaced the groups of musicians and hackers who for several summers occasionally used the top floor as a hostel for holiday courses or coding retreats, a few out of mutual fondness still come and camp in the grounds. The last car rumbles down the drive into the night, and, as it fades, a Bach partita starts to emanate from the back dining room. By the time he has finished playing, closing up will be well advanced and the patron will have unwound enough to pitch in and chivvy everyone along so that by a shade after midnight the staff are on their way home. I have slipped downstairs and put the kettle on in the room that used to be a kitchen and is now an office–cum–common-room, the award certificates just starting to outnumber the portraits of composers on the walls, so that I can take a mug of tea out to the back stairs just as my tired-but-happy friend lights up his going-to-bed cigarette. We lean against the stairs and drink our tea in silence, as nothing more needs to be said.

Wed, 07 Aug 2013 22:15:00 +0200

Older entries

Last updated 2023/03/21

Log