(The much-delayed part 2 in an ongoing series: shiny things in telepathy-glib)

Over a year ago I explained my first major round of work on telepathy-glib, with which it could be used to implement D-Bus services. I started to write this post shortly after, but then got distracted by a bee or something; recent discussion about having a D-Bus binding in GLib has finally prompted me to finish it.

The second major round of work on telepathy-glib dealt with implementing client code, and went sufficiently well that telepathy-qt4 (currently under development) uses essentially the same model.

Again, while telepathy-glib is intended for Telepathy implementors, I think TpProxy's ideas are generically useful, particularly for "large" D-Bus APIs.

Some background: dbus-glib and libtelepathy

The API provided by dbus-glib for client code is, er, rather less than ideal. The basic object is DBusGProxy, which has a few problems.

There's one per interface, rather than one per object; this leads to a maze of twisty GObjects, all suspiciously similar. Our old client library, libtelepathy, got round this by subclassing DBusGProxy to make helper objects called TpChan, TpConn etc., which were simultaneously the proxy for the "main" interface (Channel, Connection etc.), and a factory for proxies for the other interfaces (like Channel.Type.Text and Channel.Interface.Password). This still wasn't ideal, but it was a start.

If you use a DBusGProxy in any way after it has emitted the destroy signal, this is considered to be invalid use of the API, and dbus-glib asserts all over the place. We'd rather it didn't do that - remote objects becoming invalid is a fact of life, and if you don't immediately discard all your references to those objects, remote operations on them should just give you a runtime error. You have to be prepared to handle such runtime errors already, because any IPC call can fail at any time.

The main API for calling methods uses varargs and is rather subtle. Notably, the calling convention for variant parameters (of D-Bus signature 'v') is not consistent with the calling convention for any other container. For any other container, for instance a GPtrArray *, you pass in a GPtrArray ** pointing to a GPtrArray * variable containing NULL; on success, a pointer to a new GPtrArray is written into that variable. For variant parameters, though, you must pass in a GValue * pointing to a zero-filled GValue, and the result will be written into that GValue.

dbus-binding-tool generates thin wrappers around this method-calling API which provide type-safety, although they can't directly specify a non-default timeout.

As for signals, before binding to a signal, you have to tell dbus-glib about its signature. This tells dbus-glib how it should push the signal arguments onto the C stack (in GObject terminology: how it should marshal them), to match the signature of the signal-handler function.

The documentation claims that this isn't necessary for objects supporting the Introspect method, but that's untrue - on the other hand, if the documented behaviour actually happened, I would consider this to be a critical bug in dbus-glib, since it would enable remote objects to crash dbus-glib clients (by causing signal arguments to be pushed onto the stack in a way that does not match the signature of the signal handler).

Of course, the signal-adding function takes varargs, and has to be called exactly once per signal per object. If you don't, dbus-glib asserts when you connect to the signal; if you call it twice, dbus-glib asserts anyway.

TpProxy

TpProxy is a GObject subclass vaguely inspired by Python's dbus.proxy.ProxyObject. It isn't a subclass of DBusGProxy, which lets us encapsulate dbus-glib almost completely - in fact, all the references to DBusGProxy are in a separate header, proxy-subclass.h, only used by subclasses of TpProxy (as opposed to normal library API users).

TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. As explained below, I believe that this is the most natural object mapping for D-Bus interfaces; in practice it tends to lead to clearer code for us, since one remote object often maps nicely to one UI element (of course, if your objects only have one interface, this approach is no better or worse than dbus-glib's).

TpProxy provides support for the following common D-Bus things:

• Calling methods
• Connecting to signals
• Objects becoming invalid
• Objects with optional interfaces
• Extensibility

It doesn't currently provide any particular help with Properties - the Properties interface is just another interface as far as we're concerned. We find that, in practice, that's not a problem (even though we use D-Bus properties extensively).

In telepathy-qt4 we basically reinvented TpProxy in C++, as Tp::DBusProxy. I've put in some comments below illustrating how the general ideas we used in TpProxy translate into a different object mapping.

Calling methods

Method calls on TpProxy are done through auto-generated wrapper functions, much like DBusGProxy. These look something like this:

typedef void (*tp_cli_channel_callback_for_close) (TpChannel *proxy,
    /* any 'out' arguments would go here - Close() doesn't have any */
    const GError *error, gpointer user_data,
    GObject *weak_object);

TpProxyPendingCall *tp_cli_channel_call_close (TpChannel *proxy,
    gint timeout_ms,
    /* any 'in' arguments would go here - Close() doesn't have any */
    tp_cli_channel_callback_for_close callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object);

Methods for some interfaces, like DBus.Properties, are available on TpProxy itself; others, like Telepathy's Channel.anything, are only available on a particular subclass, like TpChannel.

Things to note here:

• It's an asynchronous call with a callback. General design principle: "this is IPC, get over it". We don't try to hide the fact that IPC is taking place by doing pseudo-blocking calls.

• The timeout is explicit (although you can always pass -1 to let libdbus pick a "reasonable" default for you). This is because the remote service might not respond, and API users should be at least vaguely aware of this: "this is IPC, get over it".

• The callback isn't a GCallback; it has an explicit signature, for some semblance of type-safety (it's not much, but it's better than nothing).

• The callback takes a GError, because any IPC call can fail for any reason.

• A TpProxyPendingCall * is returned. This is a pointer to an object that only exists until just after the callback is called (so you can ignore it if you don't want it), with a method you can call to cancel the method call (with hindsight this is not ideal, and if I was writing this API now, I'd use gio's GCancellable). "Cancel" is perhaps slightly misleading: the method call still takes place (it was already in libdbus's outgoing queue) but the result gets ignored and the callback isn't called. This is mostly so you an object can forget all the calls it was busy making at the time that the results become irrelevant (e.g. the object is destroyed).

• There is an extra weak_object argument, which is weakly referenced and is passed to the callback: in practice, most users of telepathy-glib use this as well as or instead of user_data. If the weak_object runs out of references, the callback is automatically cancelled, which means that in practice, explicit cancellation is almost never needed.

• The generated code has as its first argument a suitable subclass of TpProxy, in this case TpChannel (we tell the code generator about subclasses).

• You can't see it in the API, but the TpProxy gains an extra ref while a method call is in progress, so you never have to worry about what happens if the proxy gets unreffed during a method call.

• There's an explicit destructor for the user_data, which is called even if you "cancel" the method call.

In telepathy-qt4, these methods are on small generated helper classes similar to dbus-python's dbus.Interface, one per interface (we provide accessors for them on the non-generated DBusProxy subclass). Instead of using callbacks, we return a temporary QObject per call, which emits a Qt signal on success or failure (that's how all async calls in QtDBus work).

Connecting to signals

Similarly, signals have some generated functions:

typedef void (*tp_cli_channel_signal_callback_closed) (TpChannel *proxy,
    /* any arguments would go here - Closed doesn't have any */
    gpointer user_data, GObject *weak_object);
TpProxySignalConnection *tp_cli_channel_connect_to_closed (TpChannel *proxy,
    tp_cli_channel_signal_callback_closed callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object,
    GError **error);

API notes for these:

• The callback doesn't take a GError any more, because signals can't fail; the weak_object and the user_data are the same as for method calls
• This isn't actually a GObject signal at all: those aren't particularly typesafe, and lack namespacing. This might not be very binding-friendly, we haven't tried doing that yet... it might be worth introducing a signal with details, like in dbus-glib, for which these functions are "C bindings".
• The returned TpProxySignalConnection lets you disconnect from the signal, just like the TpProxyPendingCall above; it can safely be ignored
• Similarly, if the weak_object dies, then the signal automatically disconnects
• It is possible for connecting to a signal to fail: this happens if the proxy doesn't actually have the requested interface. If this happens, it's graceful, not a crash. In practice, the flow of code is almost always such that error can safely be NULL because the interfaces are already known.

In telepathy-qt4 the per-interface generated helper classes emit Qt signals. In many cases, we end up proxying these signals by responding to them in the hand-written Tp::DBusProxy subclass by emitting a different signal, in order to give them a nicer representation.

Becoming invalid

This is IPC, and any error can happen to you at any time. We found that it was useful to have a general concept of "this object is no longer useful", so we introduced the concept of an object becoming invalidated.

The invalidation reason is stored on the object as a GError. If this GError is NULL (as it is initially), then the object is still expected to be valid; if it's non-NULL, the remote object has vanished and the local proxy no longer works.

There are several ways a TpProxy can become invalid:

• its bus name falls off the bus (the process exited or crashed)
• application-specific reasons (usually triggered by a D-Bus signal in a subclass, like the Telepathy Channel's Closed signal)
• the GObject gets disposed

Method calls on an invalidated object always fail, with the invalidation reason as the error. This means that calling a method on a TpProxy should never crash your process. The method's callback has to be prepared to handle an error in any case, because this is Sparta^WIPC, so there's no loss in giving it another error.

Connecting to signals on an invalidated object always fails, and any existing signal connections are disconnected when it becomes invalidated. This means that if a new remote object appears, and it happens to have the same object path, bus name etc. as the old one, you won't get its signals.

A TpProxy can either be bound to a unique bus name or a well-known bus name. In the equivalent code in telepathy-qt4, we call this "stateful" or "stateless" - these names don't capture the intention perfectly, but they're the best we could come up with.

A proxy for a stateful API like Telepathy's Channel should always bind to a unique name - when the unique name falls off the bus, the TpProxy becomes invalidated automatically, representing the fact that the Channel you had no longer exists, and nothing can be a perfect replacement for it (the VoIP call has already been terminated, you've already left the chatroom, or whatever - if you make another Channel, that's e.g. a distinct VoIP call).

A proxy for a stateless API like Telepathy's ConnectionManager, where an exiting process can disappear, be service-activated again, and still have exactly the same API, should bind to the well-known name. Proxies for well-known names aren't invalidated when the process exits.

Qt doesn't have a natural equivalent for GError, so the invalidation reasons in telepathy-qt4 are just a pair of strings, the namespaced name and the message - basically a libdbus DBusError. The principle is the same, though.

Optional interfaces

As mentioned above, TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. I believe that this is the most natural object mapping for the D-Bus object model, because the D-Bus interfaces on a remote object can behave like any of these:

• classes and subclasses (Telepathy.Channel.Type.Text is a Telepathy.Channel)
• shared interfaces (some Telepathy.Channel objects, of many Types, implement Telepathy.Channel.Interface.Group; any D-Bus object can implement DBus.Properties)
• optional features (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.ChatStates, some do not)
• discoverable extensions (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.Messages, which is more or less Text 2.0)

The way in which interfaces are discovered is also variable. The "classic" D-Bus way to discover interfaces would be to call Introspect. Telepathy services still support that style of introspection, but we don't use it for anything beyond d-feet:

• Introspect returns a blob of XML, which you have to parse into something useful; if it was in a D-Bus data structure, you'd already have some sensible data structure.
• Finding out more information about the supported interfaces than their names isn't very useful: in a static language like C you already need to know the methods, signals, properties and their signatures in order to write and compile your code, and in a dynamic language like Python consumers of Introspect end up being remotely crashable by services with unexpected method signatures (this is one of dbus-python's big mistakes in my opinion).
• If a method that you want to call turns out to be missing, the worst that can happen (in a competently written service) is that it fails with a D-Bus error - and you have to handle D-Bus errors anyway, because this is IPC and anything could fail.
• Static bindings like dbus-glib have poor support for removing interfaces for which C code exists at runtime, whereas Telepathy needs features like an IM connection that might or might not support avatars (we don't get to find out until we've successfully connected to the server).

Instead, our older interfaces have a method on the "base class" (Channel, Connection etc.) called GetInterfaces, which just returns an array of strings. Newer interfaces (like Account), and older interfaces that have been ported (like Channel), have a property called Interfaces which is, again, an array of strings - this lets us combine the download of the interfaces list with downloading other basic information in a GetAll call.

To cater for all these ways to use interfaces, the TpProxy base class has very basic support for interface discovery - you can ask it whether it supports an interface, and you can tell it that it does, in fact, support an interface.

Asking which interfaces are supported is directly useful for library users; it's also used as a check by the generated method-call and signal-connection stubs, which return a canned error (Telepathy.Error.NotImplemented in Telepathy's case) if the object isn't known to have the interface.

Telling TpProxy that it does support an interface is intended for use by subclasses, and might have been protected if we were writing C++: what happens in practice is that a concrete subclass like TpChannel knows how to discover the fully supported interfaces for this particular object, does so, and calls methods on the TpProxy to tell it which interfaces can work.

Extensibility

The API stubs used by TpProxy to call methods and bind to signals are generated from XML documents containing an augmented form of D-Bus introspection data. These XML documents contain various Telepathy-specific extensions, but none of the extensions are language-specific - telepathy-glib, telepathy-python and telepathy-qt4 all operate from exactly the same XML specification. telepathy-qt4's different object-mapping did require us to tighten up some rules that were previously only conventions, but even so, the format is the same. I think this is vitally important - it just doesn't scale to have one set of language-specific annotations in a D-Bus API for each language that will have bindings.

One thing that we make heavy use of is what I call "Ugly_Case", which is camel-case with underscores at word boundaries. This gives us a simple, unambiguous and foolproof rule to use in code generators whenever we generate any of the popular conventions for identifiers:

• CamelCase: delete underscores
• javaCamelCase: force everything before the first underscore to lower case, then delete underscores
• lower_case: force everything to lower case, retain underscores
• UPPER_CASE: force everything to upper case, retain underscores

dbus-glib, by contrast, applies a complex and subtly buggy algorithm to the CamelCase D-Bus names, which results in our SendDTMF method being mapped into GLib as send_dt_mf, and requires that telepathy-glib's Python reimplementation of dbus-binding-tool uses a bug-for-bug compatible implementation (we ended up reimplementing dbus-binding-tool in order to generate the TpProxy-based bindings).

Our code generation tools are still rather ad-hoc, because so is our spec format, but for draft interfaces it's possible to copy them into individual projects and use them to generate additional methods for TpProxy or any subclass. This is how we implement unfinished APIs like geolocation in Empathy, for instance.

The Interfaces or GetInterfaces hook described above is entirely usable for these extension interfaces, and in fact that's how they're set up.

In telepathy-qt4, the generated method stubs are genuine C++ methods, so we can't just append methods to an existing class: this is why we have one helper class per interface, so individual clients can generate a helper class for their particular version of an unfinished API, and instantiate an object of this class attached to a particular Tp::DBusProxy.

When an interface becomes stable, it can be added to the set of interfaces for which code is generated in telepathy-glib or telepathy-qt4, at which point any client or service that was already using the final draft of that interface can easily be ported to the library version using sed (the API remains the same).

Becoming ready

The concept of being "ready" does not directly exist in TpProxy, but is implemented in TpChannel, TpConnection and TpConnectionManager. The idea is that a freshly created proxy object isn't really fully usable yet - you have to connect to basic signals and recover the initial state of the remote object, as well as checking which extension interfaces are supported. In many well-designed D-Bus APIs you can do this initial setup with a few signal connections and a DBus.Properties.GetAll call.

After doing that initial setup (which has to be done asynchronously, because it's IPC), the TpProxy subclass has a local cache of the remote object's state (not necessarily in the same format as the representation on the bus), which can be accessed synchronously at any time, and will be updated in response to change-notification signals.

TpChannel and subclasses like it have explicit support for checking for readiness, and having a callback called when the object is ready or invalidated (whichever happens first). The "ready" property also supports the GObject notify signal.

telepathy-qt4 took this concept and ran with it - there is library support for objects that can become ready, including a mixin. If I had as much time to work on telepathy-glib as I wish I had, this would be one of the first telepathy-qt4 features to be added to telepathy-glib.

Features becoming ready

Going beyond that, in an extensible framework like Telepathy it's probable that not every client understands (or wants to understand) every feature of every object, so it's highly inefficient for every proxy object to subscribe to all the change notification signals and cache all the remote state on the off-chance that the library user wants them.

In the API for TpContact (which is not actually a TpProxy for various reasons) we introduced the concept of optional features; telepathy-qt4 extended this idea throughout the library. The idea is that each library user knows which of the optional features are "interesting" to it, and makes a single asynchronous call (which expands into a series of D-Bus calls inside the library) to download the state for core functionality plus all of the selected features, and subscribe to change notification for all of those.

This is another telepathy-qt4 thing that telepathy-glib still needs to catch up with; until someone works out how to make the clone() syscall apply to programmers, I fear all libraries are doomed to lag behind how their designers want them to look :-)

(license: CC-BY-3.0 / LGPL-2.1+)

At some point I might document my current e-mail setup, but for now, here's the bit that caused plenty of frustration on Friday. Many howtos for postfix and dspam will tell you that if you embed user IDs in the signature with PgSQLUIDInSignature on (or the MySQL equivalent, you can do something like this for retraining, and set it up to receive spam@example.com:

# master.cf
dspamretrain unix -     n       n       -       10      pipe
  flags=R user=dspam argv=/usr/bin/dspam --user dspam
      --class=${nexthop} --source=error

# mailbox_transport map
spam    dspamretrain:spam
notspam dspamretrain:innocent

in which --user dspam names an arbitrary user (dspam will extract the uid from the signature and switch from "dspam" to the correct user, but the command line argument is still required, for no particular reason).

I said it named an arbitrary user, but actually that's a lie. The things to be careful of are:

• the dspam user must be listed in your database (e.g. in dspam_virtual_uids for postgres users with the default setup)
• if you're running dspam in opt-in mode, dspam must have opted in (so create /var/spool/dspam/opt-in/local/dspam.dspam)

Having failed at both of those, my retrain address wasn't working, and sadness ensued.

I recently converted most of the Debian packaging for Telepathy and related projects from bzr to git, while changing the repository contents from just the debian/ directory to the whole source tree. Here's the recipe, using the latest one to be converted (pymsn) as an example.

Create a repository

mkdir pymsn
cd pymsn
git init

Mass tarball import

• download all the tarballs into ../tarballs

• rename all the tarballs to *.orig.tar.gz:

  rename 's/^pymsn-/pymsn_/' *.tar.gz
  rename 's/\.tar\.gz$/.orig.tar.gz/' *.tar.gz
  

• use git-import-orig to import them:

  cd ../pymsn
  ls ../tarballs/pymsn_*.orig.tar.gz
  for v in 0.2 0.2.1 0.2.2 0.3.0 0.3.1 0.3.2 0.3.3; do \
  git-import-orig --no-merge --no-dch --pristine-tar \
      -u $v ../tarballs/pymsn_$v.orig.tar.gz; done
  

• throw away the resulting master branch in favour of using one we convert from bzr later:

  git checkout upstream
  git branch -D master
  

Converting the Debian packaging

This has the slight complication that in some (but not all) of the bzr commits, the repository contained only the contents of the debian directory, in the root of the repository. So, I needed to rewrite history, with this script:

#!/bin/sh
# index-filter.sh
if git ls-files -s | grep debian; then
    :
else
    git ls-files -s |
        sed -e "s-\t-&debian/-" |
        GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE
fi

• Download unstable and experimental packaging into ../pymsn-experimental and ../pymsn-unstable

• Use bzr-fast-export and git-fast-import to import the two branches:

  ~/.bazaar/plugins/fastimport/exporters/bzr-fast-export.py \
      ../pymsn-unstable | git fast-import
  
  
  git branch -m master debian-lenny
  
  
  ~/.bazaar/plugins/fastimport/exporters/bzr-fast-export.py \
      ../pymsn-experimental | git fast-import
  
  
  git branch -m master debian
  
  
  git branch -D tmp
  

• Rewrite history so changelog, control etc. are consistently inside a debian directory, with the script shown above:

  git filter-branch --index-filter $(pwd)/../index-filter.sh debian-lenny
  rm -r .git/refs/original
  
  
  git filter-branch --index-filter $(pwd)/../index-filter.sh debian
  rm -r .git/refs/original
  

• Merge the appropriate upstream versions into each branch:

  git checkout debian
  git merge upstream/0.3.3
  vim debian/control        # and change Vcs-Bzr to Vcs-Git
  dch "Move packaging to git"
  debcommit -a
  
  
  git checkout debian-lenny
  git merge upstream/0.3.1
  vim debian/control        # and change Vcs-Bzr to Vcs-Git
  dch "Move packaging to git"
  debcommit -a
  

• Rescue Debian patches and put them on a debian-patches branch, which is rebased with each upstream release. For pymsn, there aren't any patches in the debian branch at the moment:

  git branch debian-patches upstream/0.3.3
  

  but there is one in the debian-lenny branch:

  git branch debian-lenny-patches upstream/0.3.1
  git checkout debian-lenny
  cp debian/patches/00-fix-dns-resolution.diff ..
  git checkout debian-lenny-patches
  patch -p1 < ../00-fix-dns-resolution.diff
  git commit -a
  

• Make a repository on alioth:

  cd /git/pkg-telepathy
  GIT_DIR=pymsn.git git init --bare --shared
  GIT_DIR=pymsn.git git config hooks.mailinglist ...
  vim pymsn.git/description
  

• Push to the repository:

  git gc --aggressive
  git remote add origin git+ssh://git.debian.org/git/pkg-telepathy/pymsn.git
  git push --all
  

• On alioth, do some more setup after the initial push:

  cat > pymsn.git/hooks/post-receive <<EOF
  #!/bin/sh
  exec /usr/local/bin/git-commit-notice
  EOF
  chmod +x pymsn.git/hooks/post-receive
  GIT_DIR=pymsn.git git symbolic-ref HEAD refs/heads/debian
  

• Mark the old repositories as no longer used:

  cd ../pymsn-unstable
  dch $'This repository is no longer used, instead please use git:\n'\
  "git://git.debian.org/git/pkg-telepathy/pymsn.git" -c changelog
  debcommit -c changelog
  bzr push
  cd ../pymsn-experimental
  dch $'This repository is no longer used, instead please use git:\n'\
  "git://git.debian.org/git/pkg-telepathy/pymsn.git" -c changelog
  debcommit -c changelog
  bzr push
  cd ../pymsn
  

I've found myself writing this several times recently in the context of Telepathy, so I thought I'd make a blog post that I can refer people to.

D-Bus is defined (by the wire protocol spec) to be an asynchronous message-passing system, and libdbus behaves accordingly; there are no blocking calls at the wire protocol level. However, libdbus provides a "blocking" API (dbus_do_something_and_block), via a mechanism I'll refer to as "pseudo-blocking" for the purposes of this post. Most D-Bus bindings (at least dbus-glib, dbus-python and QtDBus) expose this pseudo-blocking behaviour to library users.

These pseudo-blocking calls work like this:

• send a method call message (call it M)
• while a reply to M has not been received:
  • select() or poll() on the D-Bus socket, ignoring all other I/O
  • whenever a whole message has been received:
  • check whether the message is a reply to M
  • if it is (call it R), stop
  • if it is not, put it on the incoming message queue

The messages received between M and R are delivered when the main loop is next entered.

This can cause a number of problems:

• Messages are re-ordered: messages received between M and R aren't delivered until after the reply, violating the ordering guarantee that the D-Bus daemon usually provides.

  (This causes practical problems if a signal indicating object destruction is delayed - the client gets a method reply "UnknownMethod", has to guess that this is because the object has vanished, and can't know why it vanished until the signal indicating its destruction arrives with more details.)

• The client is completely unresponsive until the service replies - if the service has somehow got wedged (e.g. telepathy-gabble is meant to be purely non-blocking and asynchronous, but there are cases where it will do blocking I/O on SSL connections due to Loudmouth bugs), the client will be unresponsive for (by default) 25 seconds until the call times out.

  (Clients shouldn't crash or lock up, whatever happens to the services they depend on.)

• The client can't parallelize calls - if a signal (e.g. an incoming Text message in Telepathy) causes method calls to be made, a client that uses pseudo-blocking calls can't even start processing the next message until those method calls return

• If two processes make pseudo-blocking calls on each other, deadlock occurs. This is particularly tricky in the presence of a plugin architecture and shared D-Bus connections - a plugin that "knows" it's a client and not a service, and a plugin in the same process that "knows" it's a service and not a client, can end up sharing a connection, resulting in a process that is both a service and a client (and hence deadlock-prone).

  (We've seen this happen in the OLPC Sugar environment and on Nokia internet tablets; it's not just a theoretical concern.)

As a result, telepathy-glib's code generation mechanism does not generate any pseudo-blocking code. There are several alternative modes of operation that we do support:

• Fully asynchronous: call a method now, pass it a callback, get your callback called from the main loop later. This can be awkward to program with, but is the only way forward for most non-trivial projects.

• Re-entrant main loop: re-enter the main loop, processing all messages (D-Bus, GUI, network, anything), and run it until the reply has been received. This is useful for trivial projects (like telepathy-glib's regression tests), but dangerous for non-trivial projects (like Empathy), and I'm somewhat regretting implementing this.

• Ignoring the reply: when appropriate (it rarely is), you can make an asynchronous call with callback = NULL and the reply will be ignored completely

We make a couple of narrowly targeted exceptions to the "no pseudo-blocking" policy in internal code, by allowing telepathy-glib internals to make a small number of pseudo-blocking method calls to the dbus-daemon (which is the one component we can definitely trust to return results promptly).

Here are some examples of the same strategies in other libraries:

• Pseudo-blocking: dbus-python method calls with no special keyword arguments; dbus-glib dbus_g_proxy_call; QDBus calls with mode QDBus::Block

• Fully asynchronous: dbus-python method calls with reply_callback and error_callback keyword arguments; dbus-glib dbus_g_proxy_begin_call; QDBusConnection::callWithCallback

• Re-entrant: QDBus calls with mode QDBus::BlockWithGui

• Ignoring reply: dbus-python method calls with ignore_reply keyword argument; QDBus calls with mode QDBus::NoWaitForReply

I've been meaning to document this for ages...

My laptop is set up with an encrypted root filesystem using LVM and dmcrypt. It also has a small Debian stable system for recovery purposes (I don't have a CD drive, so if everything goes horribly wrong, I can't just boot from a live CD).

Here's how to set up something similar:

Step 1. Set up the recovery system

Start the Debian lenny (beta) installer as usual. (I originally used etch, but these instructions are for lenny - either should work.)

When you get to "Partition disks", choose "Manual".

Here's the partitioning scheme to use:

• main partition for LVM, taking up the whole disk minus 1GB or so. select "Do not use", for now

• 1GB recovery partition at the end of the disk (this will also be the main system's /boot, since /boot needs to be unencrypted anyway). Use the defaults: ext3 mounted at /.

Finish partitioning and write changes to disk, and wait for the base system to install.

Say yes to installing Grub to the MBR for now (it might be possible to do the install more cleanly by installing Grub to the recovery partition's boot sector instead - I haven't tested that).

Reboot into the freshly installed recovery system and satisfy yourself that it works.

Step 2. Set up the main system

Now boot the Debian installer again. This time it's for your installed system.

At the "Partition disks" stage, choose "Manual" again. Select the main partition and choose "Use as: physical volume for encryption".

Next select "Configure encrypted volumes". Your main partition will now be randomized - this is slow - and some time later you'll be asked for a passphrase. You'll have to type this in at each boot.

Select the contents of the "disk" Encrypted volume (hda1_crypt) and choose Use as: physical volume for LVM.

Now "Configure the Logical Volume Manager" and create a Volume Group. I always use the laptop's hostname as the VG name (this reduces confusion if you ever plug the disk into another machine for disaster recovery).

Create a Logical Volume called swap, the size you want your swap space to be. If you plan to use suspend-to-disk, this needs to be at least as large as your RAM.

Create a Logical Volume called root, for the root filesystem. If you want separate "partitions" for things like /home, now is a good time to create them too; if you want to use my schroot howto, leave some unallocated space in the VG for that.

Set your swap LV to be used as a swap area, and your root LV to be used as ext3 mounted at /. If you wanted extra LVs, set them up too.

Also set your recovery partition to be used as ext3, mounted on /boot, and not reformatted.

It should now look something like this (smcvcrypt is the name of a KVM virtual machine in which I've been testing these instructions, normally you'd have the laptop's hostname there).

Proceed with the installation.

Install Grub to the MBR - this will temporarily make the recovery system unbootable, but shrug never mind. Finish the installation and reboot into your new main system.

Step 3. Make them dual-boot

Within the main system your recovery system is also visible, at /boot. Bind-mount /dev onto /boot/dev, chroot into /boot, and run "update-grub /dev/hda2", where hda2 is the partition where the recovery partition is. Leave the chroot.

Edit /boot/grub/menu.lst and put this right at the end:

title Go to recovery system
root (hd0,1)
chainloader +1

(Replace (hd0,1) with what Grub thinks the recovery partition is - the second number is the partition number starting from 0, so /dev/hda5 would be (hd0,4) and so on.)

Also edit /boot/boot/grub/menu.lst and put this right at the end:

title Back to main system
root (hd0)
chainloader +1

Also, still in /boot/boot/grub/menu.lst, go to the top of the file and change the colour scheme to something else (I used a red background) to indicate that this boot menu is for the recovery system.

Reboot and try it out. You should now have an extra boot menu option, "Recovery system". Selecting it will switch to the recovery system's boot menu, which has an option to switch back, and so on. Each boot menu also has some entries for kernels, any of which will boot with the appropriate root filesystem (encrypted root for the main system, unencrypted for the recovery system). Success!