Background noise
Simon McVittie
Copyright © 2008-2016 Simon McVittie
urn:uuid:0955f57e-4c2c-4614-b791-cf0bfb70f00e
Simon McVittie's software development blog
ikiwiki
2017-08-16T20:57:27Z
DebConf 17: Flatpak and Debian
http://smcv.pseudorandom.co.uk/2017/flatpak_and_debian/
<a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>
Copyright © 2017 Simon McVittie
2017-08-16T20:50:00Z
2017-08-09T17:06:00Z
<table class="img align-right"><caption>The indoor garden at Collège de Maisonneuve, the DebConf 17 venue</caption><tr><td><a href="http://smcv.pseudorandom.co.uk/2017/flatpak_and_debian/indoor-garden.jpg"><img src="http://smcv.pseudorandom.co.uk/2017/flatpak_and_debian/indoor-garden.jpg" width="640" height="480" alt="Decorative photo of the indoor garden" class="img" /></a></td></tr></table>
<p>I'm currently at DebConf 17 in Montréal, back at DebConf for the first
time in 10 years (last time was DebConf 7 in Edinburgh). It's great to
put names to faces and meet more of my co-developers in person!</p>
<p>On Monday I gave a talk entitled “<a href="https://debconf17.debconf.org/talks/59/">A Debian maintainer's guide to
Flatpak</a>”, aiming to introduce
Debian developers to Flatpak, and show
how Flatpak and Debian (and Debian derivatives like SteamOS) can help
each other. It seems to have been quite well received, with people
generally positive about the idea of using Flatpak to deliver backports
and faster-moving leaf packages (games!) onto the stable base platform
that Debian is so good at providing.</p>
<p><a href="http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/a-debian-maintainer-s-guide-to-flatpak.vp8.webm">A video of the talk</a> is available from the Debian
Meetings Archive.
I've also put up my slides in the DebConf <code>git-annex</code> repository, with
some small edits to link to more source code:
<a href="https://annex.debconf.org/debconf-share/debconf17/slides/59-debian-maintainers-guide-to-flatpak.html">A Debian maintainer's guide to Flatpak</a>.
<a href="https://git.collabora.com/cgit/user/smcv/dc17.git/">Source code for the slides</a>
is also available from Collabora's git server.</p>
<p>The next step is to take my proof-of-concept for building
Flatpak runtimes and apps from Debian and SteamOS packages,
<a href="https://git.collabora.com/cgit/user/smcv/flatdeb.git/">flatdeb</a>, get it
a bit more production-ready, and perhaps start publishing some sample
runtimes from a cron job on a Debian or Collabora server. (By the way,
if you downloaded that source right after my talk, please update - I've
now pushed some late changes that were necessary to fix the 3D drivers
for my OpenArena demo.)</p>
<p>I don't think Debian will be going quite as far as
<a href="https://endlessos.com/">Endless</a> any time soon: as
Cosimo outlined in the talk right before mine, they
deploy their Debian derivative as an immutable base OS with
<a href="https://ostree.readthedocs.io/en/latest/">libOSTree</a>, with all the
user-installable modules above that coming from Flatpak. That model is
certainly an interesting thing to think about for Debian derivatives,
though: at Collabora we work on a lot of appliance-like embedded Debian
derivatives, with a lot of flexibility during development but very limited
state on deployed systems, and Endless' approach seems a perfect fit for
those situations.</p>
<p>[Edited 2017-08-16 to fix the link for the slides, and add links for the
video]</p>
GTK hackfest 2017: D-Bus communication with containers
http://smcv.pseudorandom.co.uk/2017/dbus_and_containers/
<a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>
Copyright © 2017 Simon McVittie
2017-03-23T18:07:00Z
2017-03-23T18:07:00Z
<p>At the <a href="https://wiki.gnome.org/Hackfests/GTK2017">GTK hackfest</a> in London (which accidentally became mostly a
Flatpak hackfest) I've mainly been looking into how to make D-Bus
work better for app container technologies like <a href="http://flatpak.org/">Flatpak</a> and <a href="https://snapcraft.io/">Snap</a>.</p>
<p>The initial motivating use cases are:</p>
<ul>
<li><p><a href="https://github.com/flatpak/flatpak/wiki/Portals">Portals</a>: Portal authors need to be able to identify whether the
container is being contacted by an uncontained process (running with
the user's full privileges), or whether it is being contacted by a
contained process (in a container created by Flatpak or Snap).</p></li>
<li><p><a href="https://wiki.gnome.org/Projects/dconf">dconf</a>: Currently, a contained app either has full read/write access
to dconf, or no access. It should have read/write access to its own
subtree of dconf configuration space, and no access to the rest.</p></li>
</ul>
<p>At the moment, Flatpak runs a D-Bus proxy for each app instance that has
access to D-Bus, connects to the appropriate bus on the app's behalf,
and passes messages through. That proxy is in a container similar to the
actual app instance, but not actually the same container; it is trusted
to not pass messages through that it shouldn't pass through.
The app-identification mechanism works in practice, but is Flatpak-specific,
and has a known race condition due to process ID reuse and limitations
in the metadata that the Linux kernel maintains for <code>AF_UNIX</code> sockets.
In practice the use of X11 rather than Wayland in current systems is a much
larger loophole in the container than this race condition, but we want to
do better in future.</p>
<p>Meanwhile, Snap does its sandboxing with AppArmor, on kernels where it
is enabled both at compile-time (Ubuntu, openSUSE, Debian, Debian derivatives
like Tails) and at runtime (Ubuntu, openSUSE and Tails, but not Debian by
default). Ubuntu's kernel has extra AppArmor features that haven't yet
gone upstream, some of which provide reliable app identification via
LSM labels, which <code>dbus-daemon</code> can learn by querying its <code>AF_UNIX</code> socket.
However, other kernels like the ones in openSUSE and Debian don't have
those. The access-control (AppArmor <em>mediation</em>) is implemented in upstream
dbus-daemon, but again doesn't work portably, and is not sufficiently
fine-grained or flexible to do some of the things we'll likely want to do,
particularly in dconf.</p>
<p>After a lot of discussion with dconf maintainer <a href="https://blogs.gnome.org/desrt/">Allison Lortie</a> and
Flatpak maintainer <a href="https://blogs.gnome.org/alexl/">Alexander Larsson</a>, I think I have a plan for fixing
this.</p>
<p>This is all subject to change: see <a href="https://bugs.freedesktop.org/show_bug.cgi?id=100344">fd.o #100344</a> for the latest ideas.</p>
<h2>Identity model</h2>
<p>Each user (uid) has some <em>uncontained</em> processes, plus 0 or more
<em>containers</em>.</p>
<p>The uncontained processes include dbus-daemon itself, desktop environment
components such as gnome-session and gnome-shell, the container managers
like Flatpak and Snap, and so on. They have the user's full privileges,
and in particular they are allowed to do privileged things on the user's
session bus (like running <code>dbus-monitor</code>), and act with the user's full
privileges on the system bus. In generic information security jargon, they
are the trusted computing base; in AppArmor jargon, they are unconfined.</p>
<p>The containers are Flatpak apps, or Snap apps, or other app-container
technologies like Firejail and AppImage (if they adopt this mechanism,
which I hope they will), or even a mixture (different
app-container technologies can coexist on a single system).
They are containers (or container instances) and not "apps", because in
principle, you could install <code>com.example.MyApp</code> 1.0, run it, and while
it's still running, upgrade to <code>com.example.MyApp</code> 2.0 and run that; you'd
have two containers for the same app, perhaps with different permissions.</p>
<p>Each container has an <em>container type</em>, which is a reversed DNS
name like <code>org.flatpak</code> or <code>io.snapcraft</code> representing the
container technology, and an <em>app identifier</em>, an arbitrary non-empty
string whose meaning is defined by the container technology. For Flatpak,
that string would be another reversed DNS name like <code>com.example.MyGreatApp</code>;
for Snap, as far as I can tell it would look like <code>example-my-great-app</code>.</p>
<p>The container technology can also put arbitrary metadata on the D-Bus
representation of a container, again defined and namespaced by the
container technology. For instance, Flatpak would use some serialization
of the same fields that go in the <a href="https://manpages.debian.org/unstable/flatpak/flatpak-metadata.5.en.html">Flatpak metadata</a> file at the moment.</p>
<p>Finally, the container has an opaque <em>container identifier</em> identifying
a particular container instance. For example, launching <code>com.example.MyApp</code>
twice (maybe different versions or with different command-line options
to <code>flatpak run</code>) might result in two containers with different
privileges, so they need to have different container identifiers.</p>
<h2>Contained server sockets</h2>
<p>App-container managers like Flatpak and Snap would create an <code>AF_UNIX</code>
socket inside the container, <code>bind()</code> it to an address that will be made
available to the contained processes, and <code>listen()</code>, but not <code>accept()</code>
any new connections. Instead, they would fd-pass the new socket to the
dbus-daemon by calling a new method, and the dbus-daemon would proceed
to <code>accept()</code> connections after the app-container manager has signalled
that it has called both <code>bind()</code> and <code>listen()</code>. (See <a href="https://bugs.freedesktop.org/show_bug.cgi?id=100344">fd.o #100344</a>
for full details.)</p>
<p>Processes inside the container must not be allowed to contact the
<code>AF_UNIX</code> socket used by the wider, uncontained system - if they could,
the dbus-daemon wouldn't be able to distinguish between them and uncontained
processes and we'd be back where we started. Instead, they should have the
new socket bind-mounted into their container's <code>XDG_RUNTIME_DIR</code> and
connect to that, or have the new socket set as their
<code>DBUS_SESSION_BUS_ADDRESS</code> and be prevented from connecting to the
uncontained socket in some other way.
Those familiar with the <a href="https://www.freedesktop.org/wiki/Software/systemd/kdbus/">kdbus</a> proposals a while ago might recognise
this as being quite similar to kdbus' concept of <em>endpoints</em>, and I'm
considering reusing that name.</p>
<p>Along with the socket, the container manager would pass in the container's
identity and metadata, and the method would return a unique, opaque
identifier for this particular container instance. The basic fields
(container technology, technology-specific app ID, container ID) should
probably be added to the result of <code>GetConnectionCredentials()</code>, and
there should be a new API call to get all of those plus the
arbitrary technology-specific metadata.</p>
<p>When a process from a container connects to the contained
server socket, every message that it sends should also have the
container instance ID in a new header field. This is OK even though
<a href="https://bugs.freedesktop.org/show_bug.cgi?id=100317">dbus-daemon does not (in general) forbid sender-specified future header
fields</a>, because any dbus-daemon that supported this new feature
would guarantee to set that header field correctly, the existing
Flatpak D-Bus proxy already filters out unknown header fields, and
adding this header field is only ever a reduction in privilege.</p>
<p>The reasoning for using the sender's container instance ID (as opposed to
the sender's unique name) is for services like dconf to be able to treat
multiple unique bus names as belonging to the same equivalence class of
contained processes: instead of having to look up the container metadata
once per unique name, dconf can look it up once per container instance
the first time it sees a new identifier in a header field. For the second
and subsequent unique names in the container, dconf can know that the
container metadata and permissions are identical to the one it already saw.</p>
<h2>Access control</h2>
<p>In principle, we could have the new identification feature without adding
any new access control, by keeping Flatpak's proxies. However, in the
short term that would mean we'd be adding new API to set up a socket for
a container without any access control, and having to keep the proxies
<em>anyway</em>, which doesn't seem great; in the longer term, I think we'd
find ourselves adding a second new API to set up a socket for a container
<em>with</em> new access control. So we might as well bite the bullet and go for
the version with access control immediately.</p>
<p>In principle, we could also avoid the need for new access control by
ensuring that each service that will serve contained clients does its own.
However, that makes it really hard to send broadcasts and not have them
unintentionally leak information to contained clients - we would need to
do something more like kdbus' approach to multicast, where services know
who has subscribed to their multicast signals, and that is just not how
dbus-daemon works at the moment. If we're going to have access control
for broadcasts, it might as well also cover unicast.</p>
<p>The plan is that messages from containers to the outside world will be mediated
by a new access control mechanism, in parallel with dbus-daemon's current
support for firewall-style rules in the XML bus configuration, AppArmor
mediation, and SELinux mediation. A message would only be allowed through
if the XML configuration, the new container access control mechanism,
and the LSM (if any) all agree it should be allowed.</p>
<p>By default, processes in a container can send broadcast signals,
and send method calls and unicast signals to other processes in the same
container. They can also receive method calls from outside the
container (so that interfaces like <code>org.freedesktop.Application</code> can work),
and send exactly one reply to each of those method calls. They cannot own
bus names, communicate with other containers, or send file descriptors
(which reduces the scope for denial of service).</p>
<p>Obviously, that's not going to be enough for a lot of contained apps,
so we need a way to add more access. I'm intending this to be purely
additive (start by denying everything except what is always allowed,
then add new rules), not a mixture of adding and removing access like
the current XML policy language.</p>
<p>There are two ways we've identified for rules to be added:</p>
<ul>
<li><p>The container manager can pass a list of rules into the dbus-daemon
at the time it attaches the contained server socket, and they'll be
allowed. The obvious example is that an <code>org.freedesktop.Application</code>
needs to be allowed to own its own bus name. Flatpak apps'
implicit permission to talk to <a href="https://github.com/flatpak/flatpak/wiki/Portals">portals</a>, and <a href="https://manpages.debian.org/unstable/flatpak/flatpak-metadata.5.en.html">Flatpak metadata</a>
like <code>org.gnome.SessionManager=talk</code>, could also be added this way.</p></li>
<li><p>System or session services that are specifically designed to be used by
untrusted clients, like the version of <code>dconf</code> that Allison is working
on, could opt-in to having contained apps allowed to talk to them
(effectively making them a generalization of Flatpak portals).
The simplest such request, for something like a <a href="https://github.com/flatpak/flatpak/wiki/Portals">portal</a>,
is "allow connections from any container to contact this service"; but
for dconf, we want to go a bit finer-grained, with all containers
allowed to contact a single well-known rendezvous object path, and each
container allowed to contact an additional object path subtree that is
allocated by dconf on-demand for that app.</p></li>
</ul>
<p>Initially, many contained apps would work in the first way (and in
particular <code>sockets=session-bus</code> would add a rule that allows almost
everything), while over time we'll probably want to head towards
recommending more use of the second.</p>
<h2>Related topics</h2>
<h3>Access control on the system bus</h3>
<p>We talked about the possibility of using a very similar ruleset to control
access to the system bus, as an alternative to the XML rules found
in <code>/etc/dbus-1/system.d</code> and <code>/usr/share/dbus-1/system.d</code>. We didn't
really come to a conclusion here.</p>
<p>Allison had the useful insight that the XML rules are acting like a
firewall: they're something that is placed in front of potentially-broken
services, and not part of the services themselves (which, as with
firewalls like ufw, makes it seem rather odd when the services themselves
install rules). D-Bus system services already have total control over what
requests they will accept from D-Bus peers, and if they rely on
the XML rules to mediate that access, they're essentially rejecting that
responsibility and hoping the dbus-daemon will protect them. The D-Bus
maintainers would much prefer it if system services took responsibility
for their own access control (with or without
<a href="http://smcv.pseudorandom.co.uk/2015/why_polkit/">using polkit</a>), because fundamentally the system
service is always going to understand its domain and its intended
security model better than the dbus-daemon can.</p>
<p>Analogously, when a network service listens on all addresses and accepts
requests from elsewhere on the LAN, we sometimes work around that by
protecting it with a firewall, but the optimal resolution is to get that
network service fixed to do proper authentication and access control
instead.</p>
<p>For system services, we continue to recommend essentially this
"firewall" configuration, filling in the <code>${}</code> variables as appropriate:</p>
<pre><code><busconfig>
<policy user="${the daemon uid under which the service runs}">
<allow own="${the service's bus name}"/>
</policy>
<policy context="default">
<allow send_destination="${the service's bus name}"/>
</policy>
</busconfig>
</code></pre>
<p>We discussed the possibility of moving towards a model where the daemon uid
to be allowed is written in the <code>.service</code> file, together with an opt-in
to "modern D-Bus access control" that makes the "firewall" unnecessary;
after some flag day when all significant system services follow that pattern,
dbus-daemon would even have the option of no longer applying the "firewall"
(moving to an allow-by-default model) and just refusing to activate system
services that have not opted in to being safe to use without it.
However, the "firewall" also protects system bus clients, and services
like Avahi that are not bus-activatable, against unintended access, which
is harder to solve via that approach; so this is going to take more thought.</p>
<p>For system services' clients that follow the "agent" pattern (BlueZ,
polkit, NetworkManager, Geoclue), the correct "firewall" configuration is
more complicated. At some point I'll try to write up a best-practice for
these.</p>
<h3>New header fields for the system bus</h3>
<p>At the moment, it's harder than it needs to be to provide non-trivial
access control on the system bus, because on receiving a method call, a
service has to remember what was in the method call, then call
<code>GetConnectionCredentials()</code> to find out who sent it, then only process the
actual request when it has the information necessary to do access control.</p>
<p>Allison and I had hoped to resolve this by adding new D-Bus message
header fields with the user ID, the LSM label, and other interesting
facts for access control. These could be "opt-in" to avoid increasing
message sizes for no reason: in particular, it is not typically useful
for session services to receive the user ID, because only one user ID
is allowed to connect to the session bus anyway.</p>
<p>Unfortunately, the dbus-daemon currently lets unknown fields through
without modification. With hindsight this seems an unwise design choice,
because header fields are a finite resource (there are 255 possible header
fields) and are defined by the <a href="https://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol">D-Bus Specification</a>. The only field
that can currently be trusted is the sender's unique name, because the
dbus-daemon sets that field, overwriting the value in the original
message (if any).</p>
<p>To <a href="https://bugs.freedesktop.org/show_bug.cgi?id=100317">make it safe to rely on the new fields</a>, we would
have to make the dbus-daemon filter out all unknown header fields, and
introduce a mechanism for the service to check (during connection to
the bus) whether the dbus-daemon is sufficiently new that it does so.
If connected to an older dbus-daemon, the service would not be able
to rely on the new fields being true, so it would have to ignore
the new fields and treat them as unset. The specification is sufficiently
vague that making new dbus-daemons filter out unknown header fields is
a valid change (it just says that "Header fields with an unknown or
unexpected field code must be ignored", without specifying who must
ignore them, so having the dbus-daemon delete those fields seems
spec-compliant).</p>
<p>This all seemed fine when we discussed it in person; but GDBus already
has accessors for arbitrary header fields by numeric ID, and I'm
concerned that this might mean it's too easy for a system service
to be accidentally insecure: It would be natural (but wrong!) for
an implementor to assume that if <code>g_message_get_header (message,
G_DBUS_MESSAGE_HEADER_FIELD_SENDER_UID)</code> returned non-<code>NULL</code>, then that
was guaranteed to be the correct, valid sender uid. As a result,
<a href="https://bugs.freedesktop.org/show_bug.cgi?id=100317">fd.o #100317</a> might have to be abandoned. I think more thought is
needed on that one.</p>
<h2>Unrelated topics</h2>
<p>As happens at any good meeting, we took the opportunity of high-bandwidth
discussion to cover many useful things and several useless ones. Other
discussions that I got into during the hackfest included, in no particular
order:</p>
<ul>
<li><code>.desktop</code> file categories and how to adapt them for AppStream,
perhaps involving using the <code>.desktop</code> vocabulary but relaxing some of
the hierarchy restrictions so they behave more like "tags"</li>
<li>how to build a recommended/reference "app store" around Flatpak, aiming to
host upstream-supported builds of major projects like LibreOffice</li>
<li>how <a href="https://endlessos.com/">Endless</a> do their content-presenting and content-consuming
apps in GTK, with a lot of "tile"-based UIs with automatic resizing and
reflowing (similar to responsive design), and the applicability of similar
widgets to GNOME and upstream GTK</li>
<li>whether and how to switch GNOME developer documentation to Hotdoc</li>
<li>whether pies, fish and chips or scotch eggs were the most British lunch
available from Borough Market</li>
<li>the distinction between <a href="https://en.wikipedia.org/wiki/Stout">stout</a>,
<a href="https://en.wikipedia.org/wiki/Mild_ale">mild</a> and
<a href="https://en.wikipedia.org/wiki/Porter_%28beer%29">porter</a></li>
</ul>
<p><a href="https://wiki.gnome.org/Hackfests/GTK2017">More notes</a> are available from
the GNOME wiki.</p>
<h2>Acknowledgements</h2>
<p>The GTK hackfest was organised by <a href="https://www.gnome.org/">GNOME</a> and hosted by <a href="https://www.redhat.com/en">Red Hat</a>
and <a href="https://endlessos.com/">Endless</a>. My attendance was sponsored by <a href="https://www.collabora.com/">Collabora</a>.
Thanks to all the sponsors and organisers, and the developers and
organisations who attended.</p>
GTK Hackfest 2016
http://smcv.pseudorandom.co.uk/2016/gtk-hackfest/
<a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>
Copyright © 2016 Simon McVittie
2016-06-20T18:37:00Z
2016-06-20T18:37:00Z
<p>I'm back from the GTK hackfest in Toronto, Canada and mostly recovered from
jetlag, so it's time to write up my notes on what we discussed there.</p>
<p>Despite the hackfest's title, I was mainly there to talk about non-GUI
parts of the stack, and technologies that fit more closely in what could
be seen as the freedesktop.org platform than they do in GNOME. In particular,
I'm interested in Flatpak as a way to deploy self-contained "apps" in a
freedesktop-based, sandboxed runtime environment layered over
<a href="https://www.debian.org/">the Universal Operating System</a> and
<a href="https://wiki.debian.org/Derivatives/Census">its many derivatives</a>,
with both binary and source compatibility with other GNU/Linux distributions.</p>
<p>I'm mainly only writing about discussions I was directly involved in:
lots of what sounded like good discussion about the actual graphics toolkit
went over my head completely :-)
<a href="https://wiki.gnome.org/Hackfests/GTK2016">More notes</a>, mostly from Matthias
Clasen, are available on the GNOME wiki.</p>
<p>In no particular order:</p>
<h2>Thinking with portals</h2>
<p>We spent some time discussing Flatpak's <em>portals</em>, mostly on Tuesday.
These are the components that expose a subset of desktop functionality
as D-Bus services that can be used by contained applications: they are
part of the security boundary between a contained app and the rest of the
desktop session. Android's <em>intents</em> are a similar concept seen elsewhere.
While the portals are primarily designed for Flatpak, there's no real
reason why they couldn't be used by other app-containment solutions
such as Canonical's Snap.</p>
<p>One major topic of discussion was their overall design and layout. Most
portals will consist of a UX-independent part in Flatpak itself, together
with a UX-specific implementation of any user interaction the portal
needs. For example, the portal for file selection has a D-Bus service
in Flatpak, which interacts with some UX-specific service that will
pop up a standard UX-specific "Open" dialog — for GNOME and probably
other GTK environments, that dialog is in (a branch of) GTK.</p>
<p>A design principle that was reiterated in this discussion is that
the UX-independent part should do as much as possible, with the
UX-specific part only carrying out the user interactions that need
to comply with a particular UX design (in the GTK case, GNOME's design).
This minimizes the amount of work that needs to be redone for other
desktop or embedded environments, while still ensuring that the other
environments can have their chosen UX design. In particular, it's
important that, as much as possible, the security- and performance-sensitive
work (such as data transport and authentication) is shared between all
environments.</p>
<p>The aim is for portals to get the user's permission to carry out actions,
while keeping it as implicit as possible, avoiding an "are you sure?" step
where feasible. For example, if an application asks to open a file,
the user's permission is implicitly given by them selecting the file
in the file-chooser dialog and pressing OK: if they do not want this
application to open a file at all, they can deny permission by cancelling.
Similarly, if an application asks to stream webcam data, the UX we expect
is for GNOME's Cheese app (or a similar non-GNOME app) to appear, open
the webcam to provide a preview window so they can see what they are about
to send, but not actually start sending the
stream to the requesting app until the user has pressed a "Start" button.
When defining the API "contracts" to be provided by applications in that
situation, we will need to be clear about whether the provider is expected
to obtain confirmation like this: in most cases I would anticipate that
it is.</p>
<p>One security trade-off here is that we have to have a small amount of
trust in the providing app. For example, continuing the example of Cheese
as a webcam provider, Cheese could (and perhaps should) be a contained
app itself, whether via something like Flatpak, an LSM like AppArmor
or both. If Cheese is compromised somehow, then whenever it is running, it
would be technically possible for it to open the webcam, stream video and
send it to a hostile third-party application. We concluded that this is an
acceptable trade-off: each application needs to be trusted with the privileges
that it needs to do its job, and we should not put up barriers that are easy
to circumvent or otherwise serve no purpose.</p>
<p>The main (only?) portal so far is the file chooser, in which the contained
application asks the wider system to show an "Open..." dialog, and if the user
selects a file, it is returned to the contained application through a FUSE
filesystem, the <em>document portal</em>. The reference implementation of the UX for
this is in GTK, and is basically a <code>GtkFileChooserDialog</code>. The intention is
that other environments such as KDE will substitute their own equivalent.</p>
<p>Other planned portals include:</p>
<ul>
<li>image capture (scanner/camera)</li>
<li>opening a specified URI
<ul>
<li>this needs design feedback on how it should work for non-http(s)</li>
</ul>
</li>
<li>sharing content, for example on social networks (like Android's Sharing menu)</li>
<li>proxying joystick/gamepad input (perhaps via Wayland or FUSE, or perhaps
by modifying libraries like SDL with a new input source)</li>
<li>network proxies (<code>GProxyResolver</code>) and availability (<code>GNetworkMonitor</code>)</li>
<li>contacts/address book, probably vCard-based</li>
<li>notifications, probably based on freedesktop.org Notifications</li>
<li>video streaming (perhaps using Pinot, analogous to PulseAudio but for video)</li>
</ul>
<h2><a id="environment-variables">Environment variables</a></h2>
<p>GNOME on Wayland currently has a problem with environment variables:
there are some traditional ways to set environment variables for X11
sessions or login shells using shell script fragments (<code>/etc/X11/Xsession.d</code>,
<code>/etc/X11/xinit/xinitrc.d</code>, <code>/etc/profile.d</code>), but these do not apply to
Wayland, or to noninteractive login environments like <code>cron</code> and
<code>systemd --user</code>. We are also keen to avoid requiring a Turing-complete shell
language during session startup, because it's difficult to reason about
and potentially rather inefficient.</p>
<p>Some uses of environment variables can be dismissed as unnecessary or even
unwanted, similar to the statement in Debian Policy §9.9: "A program must not
depend on environment variables to get reasonable defaults." However,
there are two common situations where environment variables can be necessary
for proper OS integration: search-paths like <code>$PATH</code>, <code>$XDG_DATA_DIRS</code> and
<code>$PYTHONPATH</code> (particularly necessary for things like Flatpak), and
optionally-loaded modules like <code>$GTK_MODULES</code> and <code>$QT_ACCESSIBILITY</code>
where a package influences the configuration of another package.</p>
<p>There is a stopgap solution in GNOME's gdm display manager,
<code>/usr/share/gdm/env.d</code>, but this is gdm-specific and
insufficiently expressive to provide the functionality needed by
Flatpak: "set <code>XDG_DATA_DIRS</code> to its specified default value if unset,
then add a couple of extra paths".</p>
<p><code>pam_env</code> comes closer — PAM is run at every transition from "no user logged
in" to "user can execute arbitrary code as themselves" — but it doesn't
support <code>.d</code> fragments, which are required if we want distribution packages
to be able to extend search paths. <code>pam_env</code> also turns off per-user
configuration by default, citing security concerns.</p>
<p>I'll write more about this when I have a concrete proposal for how to solve it.
I think the best solution is probably a PAM module similar to <code>pam_env</code> but
supporting <code>.d</code> directories, either by modifying <code>pam_env</code> directly or
out-of-tree, combined with clarifying what the security concerns for
per-user configuration are and how they can be avoided.</p>
<h2>Relocatable binary packages</h2>
<p>On Windows and OS X, various GLib APIs automatically discover where the
application binary is located and use search paths relative to that;
for example, if <code>C:\myprefix\bin\app.exe</code> is running, GLib might put
<code>C:\myprefix\share</code> into the result of <code>g_get_system_data_dirs()</code>,
so that the application can ask to load <code>app/data.xml</code> from the data
directories and get <code>C:\myprefix\share\app\data.xml</code>. We would like
to be able to do the same on Linux, for example so that the apps in a
Flatpak or Snap package can be constructed from RPM or dpkg packages without
needing to be recompiled for a different <code>--prefix</code>, and so that other
third-party software packages like the games on Steam and gog.com can
easily locate their own resources.</p>
<p>Relatedly, there are currently no well-defined semantics for what happens
when a <code>.desktop</code> file or a D-Bus <code>.service</code> file has <code>Exec=./bin/foo</code>.
The meaning of <code>Exec=foo</code> is well-defined (it searches <code>$PATH</code>) and the
meaning of <code>Exec=/opt/whatever/bin/foo</code> is obvious. When this came up in
D-Bus previously, my assertion was that the meaning should be the same as
in <code>.desktop</code> files, whatever that is.</p>
<p>We agreed to propose that the meaning of a non-absolute path in a <code>.desktop</code>
or <code>.service</code> file should be interpreted relative to the directory
where the <code>.desktop</code> or <code>.service</code> file was found: for example, if
<code>/opt/whatever/share/applications/foo.desktop</code> says <code>Exec=../../bin/foo</code>,
then <code>/opt/whatever/bin/foo</code> would be the right thing to execute.
While preparing a mail to the freedesktop and D-Bus mailing lists proposing
this, I found that I had
<a href="https://lists.freedesktop.org/archives/xdg/2014-September/013339.html">proposed the same thing</a>
<a href="https://lists.freedesktop.org/archives/dbus/2014-September/016330.html">almost 2 years ago</a>...
this time I hope I can actually make it happen!</p>
<h2>Flatpak and OSTree bug fixing</h2>
<p>On the way to the hackfest, and while the discussion moved to topics that I
didn't have useful input on, I spent some time fixing up the Debian packaging
for Flatpak and its dependencies. In particular, I did my first upload
as a co-maintainer of <a href="https://tracker.debian.org/news/776053">bubblewrap</a>,
uploaded ostree to unstable (with the known limitation that <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=824650">the grub, dracut
and systemd integration is missing for now</a>
since I haven't been able to test it yet), got
most of the way through packaging Flatpak 0.6.5 (which I'll upload soon),
cherry-picked the right patches to make ostree compile on Debian 8 in an effort
to make backports trivial, and spent some time disentangling
<a href="https://github.com/flatpak/flatpak/commit/1d185f7dd6c4537726a86629b9ddcb516767fcb7">a flatpak test failure</a>
which was breaking the Debian package's installed-tests.
I'm still looking into
<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=827473">ostree test failures on little-endian MIPS</a>,
which I was able to reproduce on a Debian porterbox just before the end of the
hackfest.</p>
<h2>OSTree + Debian</h2>
<p>I also had some useful conversations with developers from
<a href="https://endlessm.com/">Endless</a>, who recently opened up a version of their
<a href="https://github.com/dbnicholson/deb-ostree-builder">OSTree build scripts</a>
for public access. Hopefully that information brings me a bit closer to
being able to publish a walkthrough for
<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=824649">how to deploy a simple Debian derivative using OSTree</a>
(help with that is very welcome of course!).</p>
<h2>GTK life-cycle and versioning</h2>
<p>The <a href="http://smcv.pseudorandom.co.uk/2016/gtk-versioning/">life-cycle of GTK releases</a> has already been
mentioned here and elsewhere, and there are some interesting responses
in the comments on my earlier blog post.</p>
<p>It's important to note that what we
discussed at the hackfest is only a proposal: a hackfest discussion between
a subset of the GTK maintainers and a small number of other GTK users
(I am in the latter category)
doesn't, and shouldn't, set policy for all of GTK or for all of GNOME. I
believe the intention is that the GTK maintainers will discuss the proposals
further at GUADEC, and make a decision after that.</p>
<p>As I said before, I hope that being more realistic about API and ABI
guarantees can avoid GTK going too far towards either of the possible
extremes: either becoming unable to advance because it's too constrained
by compatibility, or breaking applications because it isn't constrained
enough. The current situation, where it is meant to be compatible within
the GTK 3 branch but in practice applications still sometimes break,
doesn't seem ideal for anyone, and I hope we can do better in future.</p>
<h2>Acknowledgements</h2>
<p>Thanks to everyone involved, particularly:</p>
<ul>
<li>Matthias Clasen, who organised the hackfest and took a lot of notes</li>
<li>Allison Lortie, who provided on-site cat-herding and led us to some
excellent restaurants</li>
<li>Red Hat Inc., who provided the venue (a conference room
in their Toronto office), snacks, a lot of coffee, and several participants</li>
<li>my employers Collabora Ltd., who sponsored my travel and accomodation</li>
</ul>
GTK versioning and distributions
http://smcv.pseudorandom.co.uk/2016/gtk-versioning/
<a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>
Copyright © 2016 Simon McVittie
2016-06-14T01:56:00Z
2016-06-14T01:56:00Z
<p>Allison Lortie has provoked a lot of comment with her blog post on
<a href="https://blogs.gnome.org/desrt/2016/06/13/gtk-4-0-is-not-gtk-4/">a new proposal for how GTK is versioned</a>.
Here's some more context from the discussion at the GTK hackfest that
prompted that proposal: there's actually quite a close analogy in
how new Debian versions are developed.</p>
<p>The problem we're trying to address here is the two sides of a trade-off:</p>
<ul>
<li>Without new development, a library (or an OS) can't go anywhere new</li>
<li>New development sometimes breaks existing applications</li>
</ul>
<p>Historically, GTK has aimed to keep compatible within a major version,
where major versions are rather far apart (GTK 1 in 1998, GTK 2 in
2002, GTK 3 in 2011, GTK 4 somewhere in the future). Meanwhile, fixing
bugs, improving performance and introducing new features sometimes
results in major changes behind the scenes. In an ideal world, these
behind-the-scenes changes would never break applications; however, the
world isn't ideal. (The Debian analogy here is that as much as we
aspire to having the upgrade from one stable release to the next not break
anything at all, I don't think we've ever achieved that in practice -
we still ask users to read the release notes, even though ideally
that wouldn't be necessary.)</p>
<p>In particular, the perceived cost of doing a proper ABI break (a fully
parallel-installable GTK 4) means there's a strong temptation to make
changes that don't actually remove or change C symbols, but are clearly
an ABI break, in the sense that an application that previously worked
and was considered correct no longer works. A prominent recent example is
the theming changes in GTK 3.20: the ABI in terms of functions available
didn't change, but what happens when you call those functions changed in
an incompatible way. This makes GTK hard to rely on for applications
outside the GNOME release cycle, which is a problem that needs to
be fixed (without stopping development from continuing).</p>
<p>The goal of
<a href="https://blogs.gnome.org/desrt/2016/06/13/gtk-4-0-is-not-gtk-4/">the plan we discussed today</a>
is to decouple the latest branch of development, which moves fast and
sometimes breaks API, from the API-stable branches, which only get bug
fixes. This model should look quite familiar to Debian contributors,
because it's a lot like the way we release Debian and Ubuntu.</p>
<p>In Debian, at any given time we have a development branch
(testing/unstable) - currently "stretch", the future Debian 9. We
also have some stable branches, of which the most recent are Debian
8 "jessie" and Debian 7 "wheezy". Different users of Debian have
different trade-offs that lead them to choose one or the other of
these. Users who value stability and want to avoid unexpected changes,
even at a cost in terms of features and fixes for non-critical bugs,
choose to use a stable release, preferably the most recent; they only
need to change what they run on top of Debian for OS API changes (for
instance webapps, local scripts, or the way they interact with the GUI)
approximately every 2 years, or perhaps less often than that with the
Debian-LTS project supporting non-current stable releases. Meanwhile,
users who value the latest versions and are willing to work with a
"moving target" as a result choose to use testing/unstable.</p>
<p>The GTK analogy here is really quite close. In the new versioning model,
library users who value stability over new things would prefer to use
a stable-branch, ideally the latest; library users who want the latest
features, the latest bug-fixes and the latest new bugs would use the
branch that's the current focus of development. In practice we expect
that the latter would be mostly GNOME projects. There's been some
discussion at the hackfest about how often we'd have a new
stable-branch: the fastest rate that's been considered is a stable-branch
every 2 years, similar to Ubuntu LTS and Debian, but there's no consensus
yet on whether they will be that frequent in practice.</p>
<p>How many stable versions of GTK would end up shipped in Debian depends
on how rapidly projects move from "old-stable" to "new-stable" upstream,
how much those projects' Debian maintainers are willing to patch them
to move between branches, and how many versions the release team will
tolerate. Once we reach a steady state, I'd hope that we might have
1 or 2 stable-branched versions active at a time, packaged as separate
parallel-installable source packages (a lot like how we handle Qt).
GTK 2 might well stay around as an additional active version just
from historical inertia. The stable versions are intended to be
fully parallel-installable, just like the situation with GTK 1.2,
GTK 2 and GTK 3 or with the major versions of Qt.</p>
<p>For the "current development" version, I'd anticipate that we'd probably
only ship one source package, and do ABI transitions for one version
active at a time, a lot like how we deal with libgnome-desktop and the
evolution-data-server family of libraries. Those versions would have
parallel-installable runtime libraries but non-parallel-installable
development files, again similar to libgnome-desktop.</p>
<p>At the risk of stretching the Debian/Ubuntu analogy too far, the
intermediate "current development" GTK releases that would accompany
a GNOME release are like Ubuntu's non-LTS suites: they're more up to
date than the fully stable releases (Ubuntu LTS, which has a release
schedule similar to Debian stable), but less stable and not supported
for as long.</p>
<p>Hopefully this plan can meet both of its goals: minimize breakage for
applications, while not holding back the development of new APIs.</p>
Flatpak in Debian
http://smcv.pseudorandom.co.uk/2016/flatpak/
<a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>
Copyright © 2016 Simon McVittie
2016-06-05T11:24:00Z
2016-06-05T11:24:00Z
<p>Quite a lot has happened in xdg-app since
<a href="http://smcv.pseudorandom.co.uk/2016/xdg-app/">last time I blogged about it</a>. Most noticeably, it isn't called
xdg-app any more, having been renamed to <a href="http://flatpak.org/">Flatpak</a>.
It is now available in Debian experimental under that name, and the
xdg-app package that was briefly there has been removed. I'm currently
in the process of updating Flatpak to the latest version 0.6.4.</p>
<p>The privileged part has also spun off into a separate project,
<a href="https://github.com/projectatomic/bubblewrap">Bubblewrap</a>, which recently
had its first release (0.1.0). This is intended as a common component
with which unprivileged users can start a container in a way that
won't let them escalate privileges, like a more flexible version of
<a href="https://git.gnome.org/browse/linux-user-chroot/">linux-user-chroot</a>.</p>
<p>Bubblewrap has also been made available in Debian, maintained by Laszlo
Boszormenyi (also maintainer of linux-user-chroot). Yesterday I sent a patch
to <a href="https://bugs.debian.org/826358">update Laszlo's packaging for 0.1.0</a>.
I'm hoping to become a co-maintainer to upload that myself, since I suspect
Flatpak and Bubblewrap might need to track each other quite closely.
For the moment, Flatpak still uses its own internal copy of Bubblewrap,
but <a href="https://bugs.debian.org/824647">I consider that to be a bug</a> and
I'd like to be able to fix it soon.</p>
<p>At some point I also want to experiment with using Bubblewrap to
sandbox some of the game engines that are packaged in Debian: networked
games are a large attack surface, and typically consist of the sort
of speed-optimized C or C++ code that is an ideal home for security
vulnerabilities. I've already made some progress on jailing game engines
with AppArmor, but making sensitive files completely invisible to the
game engine seems even better than preventing them from being opened.</p>
<p>Next weekend I'm going to be heading to Toronto for the
<a href="https://wiki.gnome.org/Hackfests/GTK2016">GTK Hackfest</a>, primarily to to
talk to GNOME and Flatpak developers about their plans for sandboxing,
portals and Flatpak. Hopefully we can make some good progress there: the more I
know about the state of software security, the less happy I am with random
applications all being equally privileged. Newer display technologies like
Wayland and Mir represent an opportunity to plug <a href="https://mjg59.dreamwidth.org/42320.html">one of the largest
holes in typical application containerization</a>,
which is a major step in bringing sandboxes like Flatpak and Snap from
proof-of-concept to a practical improvement in security.</p>
<p>Other next steps for Flatpak in Debian:</p>
<ul>
<li>To get into the next stable release (Debian 9), Flatpak needs to move
from <code>experimental</code> into <code>unstable</code> and <code>testing</code>. I've taken the
first step towards that by uploading <code>libgsystem</code> to unstable.
Before Flatpak can follow, OSTree also needs to move.</li>
<li>Now that it's in Debian, please <a href="https://www.debian.org/Bugs/Reporting">report bugs in the usual Debian
way</a> or send patches to fix
bugs: <a href="http://bugs.debian.org/src:flatpak">Flatpak</a>,
<a href="http://bugs.debian.org/src:ostree">OSTree</a>,
<a href="http://bugs.debian.org/src:libgsystem">libgsystem</a>.</li>
<li>In particular, there are some OSTree bugs tagged <code>help</code>. I'd
appreciate contributions to the OSTree packaging from people
who are interested in using it to deploy <code>dpkg</code>-based operating systems -
I'm primarily looking at it from the Flatpak perspective, so the boot/OS
side of it isn't so well tested. Red Hat have <code>rpm-ostree</code>,
and I believe <a href="https://endlessm.com/">Endless</a> do something analogous
to build OS images with <code>dpkg</code>, but I haven't had a chance to look into
that in detail yet.</li>
<li>Co-maintainers for <a href="https://tracker.debian.org/pkg/flatpak">Flatpak</a>,
<a href="https://tracker.debian.org/pkg/ostree">OSTree</a>,
<a href="https://tracker.debian.org/pkg/libgsystem">libgsystem</a> would also
be very welcome.</li>
</ul>