I've been meaning to document this for ages...

My laptop is set up with an encrypted root filesystem using LVM and dmcrypt. It also has a small Debian stable system for recovery purposes (I don't have a CD drive, so if everything goes horribly wrong, I can't just boot from a live CD).

Here's how to set up something similar:

Step 1. Set up the recovery system

Start the Debian lenny (beta) installer as usual. (I originally used etch, but these instructions are for lenny - either should work.)

When you get to "Partition disks", choose "Manual".

Here's the partitioning scheme to use:

• main partition for LVM, taking up the whole disk minus 1GB or so. select "Do not use", for now

• 1GB recovery partition at the end of the disk (this will also be the main system's /boot, since /boot needs to be unencrypted anyway). Use the defaults: ext3 mounted at /.

Finish partitioning and write changes to disk, and wait for the base system to install.

Say yes to installing Grub to the MBR for now (it might be possible to do the install more cleanly by installing Grub to the recovery partition's boot sector instead - I haven't tested that).

Reboot into the freshly installed recovery system and satisfy yourself that it works.

Step 2. Set up the main system

Now boot the Debian installer again. This time it's for your installed system.

At the "Partition disks" stage, choose "Manual" again. Select the main partition and choose "Use as: physical volume for encryption".

Next select "Configure encrypted volumes". Your main partition will now be randomized - this is slow - and some time later you'll be asked for a passphrase. You'll have to type this in at each boot.

Select the contents of the "disk" Encrypted volume (hda1_crypt) and choose Use as: physical volume for LVM.

Now "Configure the Logical Volume Manager" and create a Volume Group. I always use the laptop's hostname as the VG name (this reduces confusion if you ever plug the disk into another machine for disaster recovery).

Create a Logical Volume called swap, the size you want your swap space to be. If you plan to use suspend-to-disk, this needs to be at least as large as your RAM.

Create a Logical Volume called root, for the root filesystem. If you want separate "partitions" for things like /home, now is a good time to create them too; if you want to use my schroot howto, leave some unallocated space in the VG for that.

Set your swap LV to be used as a swap area, and your root LV to be used as ext3 mounted at /. If you wanted extra LVs, set them up too.

Also set your recovery partition to be used as ext3, mounted on /boot, and not reformatted.

It should now look something like this (smcvcrypt is the name of a KVM virtual machine in which I've been testing these instructions, normally you'd have the laptop's hostname there).

Proceed with the installation.

Install Grub to the MBR - this will temporarily make the recovery system unbootable, but shrug never mind. Finish the installation and reboot into your new main system.

Step 3. Make them dual-boot

Within the main system your recovery system is also visible, at /boot. Bind-mount /dev onto /boot/dev, chroot into /boot, and run "update-grub /dev/hda2", where hda2 is the partition where the recovery partition is. Leave the chroot.

Edit /boot/grub/menu.lst and put this right at the end:

title Go to recovery system
root (hd0,1)
chainloader +1

(Replace (hd0,1) with what Grub thinks the recovery partition is - the second number is the partition number starting from 0, so /dev/hda5 would be (hd0,4) and so on.)

Also edit /boot/boot/grub/menu.lst and put this right at the end:

title Back to main system
root (hd0)
chainloader +1

Also, still in /boot/boot/grub/menu.lst, go to the top of the file and change the colour scheme to something else (I used a red background) to indicate that this boot menu is for the recovery system.

Reboot and try it out. You should now have an extra boot menu option, "Recovery system". Selecting it will switch to the recovery system's boot menu, which has an option to switch back, and so on. Each boot menu also has some entries for kernels, any of which will boot with the appropriate root filesystem (encrypted root for the main system, unencrypted for the recovery system). Success!

Developers seem to do a lot of fighting with bleeding-edge libraries installed in a --prefix, so I thought I'd share how I develop telepathy-glib (a library) and telepathy-gabble (a program using that library).

~/Collabora/telepathy/tpglib on my laptop is a git clone of telepathy-glib.git, branched using the process I documented on http://telepathy.freedesktop.org/wiki/Git, which is basically:

• clone the upstream repository (git+ssh://git.collabora.co.uk/git/telepathy-glib.git in this case) into your home directory on the server (git+ssh://people.collabora.co.uk/home/smcv/public_html/git/telepathy-glib-smcv.git in this case)
• clone that onto your laptop
• on your laptop, add "upstream" as a remote, tracking the upstream repository
• this reduces mistakes, because "git push" pushes to origin (which is in my home directory) for review, and "git push upstream" is necessary when I actually want to commit to the upstream repository

~/Collabora/telepathy/gabble is a git clone of telepathy-gabble.git, using the same process.

In my Gabble checkout, I used to configure like this:

cd ~/Collabora/telepathy/gabble
TP_GLIB_CFLAGS="-I${HOME}/Collabora/telepathy/tpglib" \
TP_GLIB_LIBS="${HOME}/Collabora/telepathy/tpglib/telepathy-glib/libtelepathy-glib.la" \
sh ./autogen.sh

This would make libtool link Gabble against the copy of telepathy-glib built by my bleeding-edge git checkout, without ever needing to install it. This allowed me to try out my latest telepathy-glib changes with Gabble's more extensive regression tests, which was a great win.

It also meant I could develop Gabble branches that required a not-yet-released telepathy-glib, and all without touching my system copy of telepathy-glib (the one I use to run Empathy and talk to people - which is the packaged version from Debian unstable or experimental).

The reason I could do this is that I laid out the telepathy-glib source tree in The Right Way™, which is: if a header is designed to be used via #include <my-library/foo.h>, then it goes in a my-library/ directory in the source tree, and includes other headers (even from the same project) with #include <my-library/other.h>.

This means you can just add the top directory of the source tree to your include path (which automake does by default) and the right thing will happen.

(Actually, this is a slight simplification of the truth: because telepathy-glib contains generated headers, you have to add the top directory of the build tree to your include path too, if you want to support out-of-tree builds. Which, of course, you do.)

Other projects that do this correctly include GLib and Avahi. Avahi is a particularly interesting example because it has numerous libraries in the same tarball (including avahi-common, avahi-core, avahi-glib and avahi-gobject, which form a dependency chain).

Projects that don't do this correctly include telepathy-mission-control (which is on my hit list for further swamp-draining) and libtelepathy (which is obsolete).

Today I committed a patch to telepathy-glib, inspired by GStreamer, to generate telepathy-glib-uninstalled.pc. The presence of this patch simplifies the process further, to:

PKG_CONFIG_PATH=$HOME/Collabora/tpglib/telepathy-glib sh ./autogen.sh

This has the same benefits as setting TP_GLIB_CFLAGS and TP_GLIB_LIBS, but is easier to remember, and is more future-proof against changes to telepathy-glib.

It only works because pkg-config has a special case for this situation - if you ask for the package "foo", pkg-config will always look for foo-uninstalled.pc before foo.pc.

From the "doing sysadmin over wobbly 3G while waiting for our plane to be allowed to take off" department comes this bit of mailing list setup. By his own admission, Rob is better at configuring Postfix than he is at making blog posts, so I get to be the one posting this...

09:58 <@Robot101> my postfix -> eoc integration is basically ninja, if I might 
                  say so myself
09:58 <@Robot101> so, an eoc transport in master.cf:
09:58 <@Robot101> eoc       unix  -       n       n       -       10      pipe 
                  user=list argv=/usr/bin/enemies-of-carlotta --quiet 
                  --incoming --sender=${sender} --recipient=${recipient}
09:59 <@Robot101> then in main.cf:
09:59 <@Robot101> eoc_destination_recipient_limit = 1
09:59 <@Robot101> virtual_mailbox_domains = lists.collabora.co.uk
09:59 <@Robot101> virtual_mailbox_maps = pcre:/etc/postfix/list_transports
09:59 <@Robot101> transport_maps = pcre:/etc/postfix/list_transports
09:59 <@Robot101> then a script/cron:
09:59 <@Robot101> su -c "enemies-of-carlotta --show-lists" list | sed 
                  's,\(.*\)@\(.*\),/^\1(-[^@]*)?@\2$/ eoc,' 
                  >/etc/postfix/list_transports
09:59 <@Robot101> giving list_transports like this:
09:59 <@Robot101> /^test(-[^@]*)?@lists.collabora.co.uk$/ eoc
09:59 <@Robot101> (thanks to smcv for assistance with regexp-generation regexp)
...
10:01 <@wjt> ten points for using the best -named mlm
10:01 <@Robot101> 2000 points for not having a mailing list system held 
                  together with procmail, shell, duct tape, gash and string

Here's a script I use to speed up compilation at work. Since I work at Collabora, I call it collaboratively. It's a prefix to a command (used in the same way as sudo or nice) which sets up icecc (Icecream) and ccache to work nicely together.

#!/bin/sh
# ~/bin/collaboratively
# Typical usage: collaboratively make check
PATH=/usr/lib/ccache:/usr/lib/icecc/bin$(echo :$PATH | sed -e s@:/usr/lib/ccache@@g)
export PATH
MAKEFLAGS='-j -l3'
export MAKEFLAGS
exec "$@"

Some more explanation of what it does:

• First, I want the gcc wrappers (actually symlinks to ccache) in /usr/lib/ccache to be invoked. This uses ccache to make sure I don't actually recompile source code that hasn't changed.

• Next, I want to use icecc, which distributes builds around the office. We run an icecc scheduler on the office server, which also contributes two CPU cores and lots of RAM to the compilation effort.

• To prevent infinite recursion between icecc and ccache, I have to make sure ccache isn't in my $PATH for a second time! My bash and zsh dotfiles automatically set up ccache, so in practice it does need editing out.

• When distributing builds, make should be parallelizing as many compiles as feasible - I use the flags -j -l3 to run an unlimited number of parallel compilation processes, but stop when the load average reaches 3 (the number of CPU cores I have, plus 1, seems a reasonable maximum load-average).

apt-get install icecc-monitor and run icemon to see whether this is all working - if it is, you should see compile jobs going off to other hosts.

(Written on Friday 16th, posted on Monday)

As I write this, I'm on a plane off the coast of the Netherlands, on the way back from a couple of days designing APIs with Rob McQueen and the RTCom people at NRC Helsinki (who we work with on the chat and VoIP functionality for Nokia internet tablets).

We've had a very useful couple of days, mainly designing the next-generation channel requesting and dispatch API for Telepathy (the same APIs I've been doing preparatory work for on the Telepathy mailing list in the last couple of weeks). API sketches for review and comment are either on are on Merge Monkey or on the way there.

Before even reaching Nokia, Rob and I did a lot of design on the plane over to Helsinki, and I took some notes: here they are, by way of a glimpse at how the Telepathy design process works when we don't have a whiteboard[1] :-)

• Geoloc looks good in principle, needs some more work
  • add more docstrings
  • add a way to ask people for uncached info (RequestLocations)
  • move access control to Co.I.Presence, call it RichPresenceAccessControl?
• DeliveryReporting looks good, ish
  • improve wording of delivery-echo and rationale
  • when sending a delivery report, add a reason for the status (using Channel_Text_Send_Error?)
• Uniqueness requirement for (channel, handle_type != 0, handle) is in the way - let's drop it
• MSN should use "authentic" 1-1 chats, and use Ch.I.UpgradeableToRoom (name tbd) to migrate the underlying switchboard to be a room
• XMPP should use Ch.I.UpgradeableToRoom too, but the 1-1 channel may also stay open
• Mutable handle => handle 0 must stay (for requestotron's benefit)
• bundles ftw
• renaming: we want SelfHandleChanged in the core
• TpC* - add ability to subscribe to ifaces ay construct time (blocks Ready) or later
• Dispatching: when registering a c'handler, specify everything it can handle
• If a new bundle can be handled in its entirety by the same handler (call it 'the bundle handler'), do it. Otherwise split it up completely (there is no 'bundle handler').
• If a channel in an existing bundle can be handled by the bundle handler, send it there, else send to the default handler.
• Group - add SelfHandleChanged, maybe? Also Connection
• Add HandleOwners: prop a{uu}, HandleOwnersChanged(a{uu}, au)
• Use TargetHandleType, TargetHandle
• Stringified initiator handle as a separate property
• Maybe even a FirstMessage property, although this stretches the definition of "immutable"
• org.freedesktop.Telepathy.Client interface

Nokia's Mikhail Zabaluev kindly took notes for some further discussion we had with him and Alberto Mardegan about the dispatch API, which he already posted to the mailing list.

We hope this API will make it possible to integrate Telepathy into the desktop better - it should make it a lot easier to share connections between processes in a useful way and hand out incoming channels to the appropriate handlers, which was always a major goal for Telepathy. It should also let us and others implement exciting new notification mechanisms (hi to the people on IRC and the mailing list wanting to patch their media players to integrate with Telepathy! :-)

I'm now a full Debian developer! Slightly too late to vote for my landlord as project leader, but it's OK, he got in anyway :-)

Thanks to the NM team, the newly delegated keyring maintainer, and the previous DPL's eleventh-hour redelegation for making this happen!

(Part 1 of an ongoing series: shiny things in telepathy-glib)

16:46 < epmf> tp-glib is made of magic

—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Some background: service-side code generation

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong.

dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time.

Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks.

Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:

• copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
• preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
• run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
• three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
• pray that the diff process hadn't randomly exchanged functions' implementations
• update the resulting code in src so it worked
• check the whole mess back into darcs
• hope that nobody else had made changes that would result in darcs conflicts

If you haven't yet gathered, this was a bit of a nightmare.

Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.

commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Frécinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200

    Interface support for bindings.

Daf recently described a large part of my job, in diagrammatic form:

(source)

Here's a handy incantation I use with the Intel X.org graphics driver on my laptop.

Put this script somewhere convenient:

#!/bin/sh
if xrandr | grep '^VGA connected' >/dev/null
then
    #echo "VGA connected"
    xrandr --output VGA --above LVDS --auto
    xrandr --output LVDS --auto
else
    #echo "VGA not connected"
    xrandr --auto
fi 2>&1 | logger -t xrandr-hotkey

and bind it to your "switch video mode" key, e.g. Fn+F7 on my Thinkpad X60s. For instance, I run xbindkeys, so I saved it as autoxrandr and put this in ~/xbindkeysrc:

"exec /home/smcv/bin/autoxrandr"
    XF86Display

Now if you press the "switch video mode" key with a monitor plugged in, you'll get the external monitor appearing above the laptop panel (my preferred configuration - I've balanced my monitor on an N800 box to get the bottom of the screen level with the top of my laptop's screen). If you press that key again after unplugging the VGA cable, the laptop will turn off the VGA output and pull all the windows onto the built-in screen.