(Part 1 of an ongoing series: shiny things in telepathy-glib)
16:46 < epmf> tp-glib is made of magic
Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).
While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.
Introduction to telepathy-glib
telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-)
My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September.
This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months.
The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!
Some background: service-side code generation
Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong.
dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time.
Early versions of telepathy-gabble used dbus-binding-tool plus a
gengobject.py to generate API stubs for the exported
objects; developers then filled in the blanks, and hey presto, we had a
GObject on D-Bus. This was fine up to a point, but had a couple of major
Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:
copy the D-Bus introspection XML from the telepathy-spec repository into a directory
xml/pristine(actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory
dbus-binding-tooland a Python script
gengobject.pyto generate C source in
three-way-diff the old version of
xml/src, the new version of
xml/src, and the real C code in
srcto create a new version of
pray that the diff process hadn't randomly exchanged functions' implementations
update the resulting code in
srcso it worked
check the whole mess back into darcs
hope that nobody else had made changes that would result in darcs conflicts
If you haven't yet gathered, this was a bit of a nightmare.
Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.
How telepathy-glib fixes code generation
telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib:
commit 355ef78d98d6fc65715845d56232199162cab12a Author: Steve Frécinaux <steve istique net> Date: Thu Aug 17 11:48:20 2006 +0200 Interface support for bindings.
Instead of running dbus-binding-tool or creating GObjects with D-Bus
interfaces, we put the entire Telepathy spec through
glib-ginterface-gen.py, a distant descendant of Gabble's
For each interface in the spec, we generate a GInterface that mirrors the
D-Bus interface. Any GObject that implements the GInterface automatically
gets the D-Bus interface if it's exported onto D-Bus.
There are a few non-obvious refinements, though:
The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns.
For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened.
This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done).
For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.
The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.
If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.
While we're generating code anyway, we generate some static inline wrapper functions which wrap
dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().
Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.
Not all interfaces are stable enough to be included in telepathy-glib's stable
API and ABI, so some of our other projects include a copy of the telepathy-glib
code-generation tools, and generate their own "mini-telepathy-glib"
(traditionally in a
/extensions/ directory) for their
implementation-specific (drafts of) interfaces. This isn't yet as polished as
it ought to be (mainly because we don't want to freeze the
augmented-introspection-XML format that we write the Telepathy spec in, because
it's rather ad-hoc and hackish in places) but it works quite well in practice.
Mixins and base classes
Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces.
There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does.
The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P
The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.