Allison Lortie has provoked a lot of comment with her blog post on a new proposal for how GTK is versioned. Here's some more context from the discussion at the GTK hackfest that prompted that proposal: there's actually quite a close analogy in how new Debian versions are developed.
The problem we're trying to address here is the two sides of a trade-off:
- Without new development, a library (or an OS) can't go anywhere new
- New development sometimes breaks existing applications
Historically, GTK has aimed to keep compatible within a major version, where major versions are rather far apart (GTK 1 in 1998, GTK 2 in 2002, GTK 3 in 2011, GTK 4 somewhere in the future). Meanwhile, fixing bugs, improving performance and introducing new features sometimes results in major changes behind the scenes. In an ideal world, these behind-the-scenes changes would never break applications; however, the world isn't ideal. (The Debian analogy here is that as much as we aspire to having the upgrade from one stable release to the next not break anything at all, I don't think we've ever achieved that in practice - we still ask users to read the release notes, even though ideally that wouldn't be necessary.)
In particular, the perceived cost of doing a proper ABI break (a fully parallel-installable GTK 4) means there's a strong temptation to make changes that don't actually remove or change C symbols, but are clearly an ABI break, in the sense that an application that previously worked and was considered correct no longer works. A prominent recent example is the theming changes in GTK 3.20: the ABI in terms of functions available didn't change, but what happens when you call those functions changed in an incompatible way. This makes GTK hard to rely on for applications outside the GNOME release cycle, which is a problem that needs to be fixed (without stopping development from continuing).
The goal of the plan we discussed today is to decouple the latest branch of development, which moves fast and sometimes breaks API, from the API-stable branches, which only get bug fixes. This model should look quite familiar to Debian contributors, because it's a lot like the way we release Debian and Ubuntu.
In Debian, at any given time we have a development branch (testing/unstable) - currently "stretch", the future Debian 9. We also have some stable branches, of which the most recent are Debian 8 "jessie" and Debian 7 "wheezy". Different users of Debian have different trade-offs that lead them to choose one or the other of these. Users who value stability and want to avoid unexpected changes, even at a cost in terms of features and fixes for non-critical bugs, choose to use a stable release, preferably the most recent; they only need to change what they run on top of Debian for OS API changes (for instance webapps, local scripts, or the way they interact with the GUI) approximately every 2 years, or perhaps less often than that with the Debian-LTS project supporting non-current stable releases. Meanwhile, users who value the latest versions and are willing to work with a "moving target" as a result choose to use testing/unstable.
The GTK analogy here is really quite close. In the new versioning model, library users who value stability over new things would prefer to use a stable-branch, ideally the latest; library users who want the latest features, the latest bug-fixes and the latest new bugs would use the branch that's the current focus of development. In practice we expect that the latter would be mostly GNOME projects. There's been some discussion at the hackfest about how often we'd have a new stable-branch: the fastest rate that's been considered is a stable-branch every 2 years, similar to Ubuntu LTS and Debian, but there's no consensus yet on whether they will be that frequent in practice.
How many stable versions of GTK would end up shipped in Debian depends on how rapidly projects move from "old-stable" to "new-stable" upstream, how much those projects' Debian maintainers are willing to patch them to move between branches, and how many versions the release team will tolerate. Once we reach a steady state, I'd hope that we might have 1 or 2 stable-branched versions active at a time, packaged as separate parallel-installable source packages (a lot like how we handle Qt). GTK 2 might well stay around as an additional active version just from historical inertia. The stable versions are intended to be fully parallel-installable, just like the situation with GTK 1.2, GTK 2 and GTK 3 or with the major versions of Qt.
For the "current development" version, I'd anticipate that we'd probably only ship one source package, and do ABI transitions for one version active at a time, a lot like how we deal with libgnome-desktop and the evolution-data-server family of libraries. Those versions would have parallel-installable runtime libraries but non-parallel-installable development files, again similar to libgnome-desktop.
At the risk of stretching the Debian/Ubuntu analogy too far, the intermediate "current development" GTK releases that would accompany a GNOME release are like Ubuntu's non-LTS suites: they're more up to date than the fully stable releases (Ubuntu LTS, which has a release schedule similar to Debian stable), but less stable and not supported for as long.
Hopefully this plan can meet both of its goals: minimize breakage for applications, while not holding back the development of new APIs.
Hi Simon! I think you gave the right example with respect to parallel installations with Qt (although we actually needed some ugly hacks in between). But for the sake of clarity I think it's good to mention that Qt doesn't breaks API+ABI in their major versions, normally ~10 years apart from each other.
If you build an app with Qt 4.0.0 it would still be able to run with Qt 4.8, the latest in the series which happened ~10 years later. Same goes for Qt5. The only thing that we don't warrant with Qt is downgrading the Qt version: if you build an app with Qt 5.4 you will surely not be able to run it against Qt 5.0.
Hope that makes thing clearer :)
Thanks to you all for thinking about these things, they are important issues. Sorry that this gets a bit rambly.
I already mentioned some of my concerns in a conversation on IRC, but I want to write them down here - and maybe you want to respond.
It feels like the plan makes sense as far as there is a stable target and an unstable one which application developers can choose between depending on what they are willing to sign up for. It's probably not controversial to say that things are unlikely to end up being that neat in reality. There will definitely be unstable-using applications (maybe a third party developer needed a feature in GTK 4.2 but is now busy for a short while and its API broke in 4.4) that either stop being developed completely, or whose maintainer (either in GNOME or out of it - even within GNOME inactive maintainers will mean things get broken from time to time until the build fixers show up, unless a policy like 'no GTK release until all modules build with it').
When a new GNOME release which depends on an unstable GTK+ happens, and distributors want to ship this, the new policy says this is a transition every time. Fine, we know how to deal with those in distributions - but, at least for Debian and Ubuntu, it means that every reverse dependency needs to be ported or removed. Each of the (temporarily or not) unmaintained ones are going to be a drag on this, and a burden on GTK+'s distribution maintainer - they will analyse the breakages and either try to fix or remove the package. That's a drag in both mental effort and time, and one that this scheme will give distributors every 6 months if they want to ship current GNOME. To get around this, it seems to me that distribution maintainers will have to adopt a 'remove first' policy so that they don't become burned out. This is where we have to hope that people don't use the unstable GTK+.
Should LTS and stable Debian ship the GTK and GNOME that correspond to the current-stable release at the time? Let's say that Debian release X freezes when GTK 5.2 (unstable) and its corresponding GNOME is current. Would you expect the GNOME maintainers to avoid packaging 5.2? To avoid updating GNOME to this release but still package GTK? Or to package all of it? If the latter, what is the maintenance commitment of the GTK+ team to this unstable-but-stable-in-the-GNOME-sense branch? Can we expect bugfixes to be cherry-picked for a few months, as normal, even if the API has moved on in master and is shipping such an intermediate ("non-LTS" in your analogy" version in an LTS/stable advisable?
Finally, themes. GTK 3 below 3.20 never promised stability in the theme API, and theme authors certainly have had to catch up with GTK's changes every cycle. Because of this, maintaining a theme was an onerous ongoing commitment. 3.20 was supposed to be the last big break, and after this the theme API was supposed to be stable - Matthias blogged about this. Now, with continual breaks, it seems that this promise is basically out of the window. As a theme author I again need to track the breaking releases of GTK and potentially port my theme each time. It feels like we had just managed to get there - the next stable release after the promise isn't even out yet - and now the rug is being pulled away again. Having more parallel-installable releases out there actually makes the situation a little bit worse since theme authors will have to maintain branches for each stable release until its obsolescence. You might think "well, having to rewrite your theme once every 2 years instead of every 6 months is an improvement", but that ignores the promise as previously mentioned - and it's not right to say that it is every 2 years. If you want your theme to be usable on current GNOME, based on unstable GTK, you need to be developing your theme for that. So the commitment is still an ongoing one.
I hope these comments are helpful at outlining some concerns of a downstream person. Happy hackfest. :)
But you/they did break API+ABI between Qt 3 and Qt 4, and then again between Qt 4 and Qt 5, right? That's the parallel I was trying to draw.
This proposal does have more frequent major versions than Qt, which is a trade-off: it means there are potentially more versions "in the wild" at any given time, but it hopefully also reduces the need for projects that don't closely track GTK and GNOME to depend on the version that is the current focus of development, reducing the probability of a change breaking those projects.
If GTK contributors want to support a particular branch for 10 years, they are of course welcome to do so - GTK 2 is still maintained, although it's really showing its age by now.
This touches on the reason that I think it's necessary to be willing to break ABI sometimes, and not necessarily a bad thing as long as you're honest about it (by bumping the SONAME). It is always possible to avoid breaking ABI, but taking that too far can easily make new features or bug-fixes prohibitively expensive, either in paid contributors' time (direct financial cost) or volunteers' time (not a direct financial cost, but also a finite resource).
I used to work on Telepathy, which tried very hard not to break ABI, and suffered for it: paid developers adding new features found it hard to be cost-effective, while volunteer community members were put off by the amount of time and effort that it took to get anything from idea to merge. Telepathy 1.0 (the first version to do a "hard" ABI break since the very early days of the project) was started but never finished, partly as a result of this high cost.
We did touch on this during discussions at the hackfest. Someone (sorry, I forget who) made the point that within GNOME, if a module becomes unmaintained-in-practice, it's considered valid for anyone else in GNOME to land fixes. That hopefully mitigates the problem within GNOME.
Outside GNOME, I think the recommendation would likely be that maintainers shouldn't get on the unstable train unless they are willing to ride it all the way to the next station :-) With semi-frequent stable releases, sticking to a stable release puts you much less far behind than in GTK's historical schedule.
To some extent, Debian already has that policy via autoremovals. If dependent packages don't want to be autoremoved, it is up to the maintainer of the dependent package to fix it - this sort of thing is why we call ourselves Debian maintainers, not Debian packagers.
This would be a decision for the GTK and GNOME maintainers based on how much work it is and how much time is available, but if it was up to me, I'd expect that we'd package both GTK 4.0 and GTK 5.2, together with the version of GNOME that corresponds to 5.2.
I would hope so. Note that the stable branches already largely exist for distribution maintainers' benefit, and as far as I can see, what goes into those branches is often already driven by what distribution maintainers want those branches to look like.
It's possible that the "LTS" branches might become even more conservative in what they will accept, taking only changes with a particularly good ratio of severity of bugs fixed / (probability × severity) of regressions expected, a lot like a Debian stable release.
That's a valid concern, and I don't know what the policy will be regarding themes.
I suspect that in practice theming is always going to be "interesting times" territory, because a theme potentially needs to know about every widget, and it's easy for a relatively minor bug to make a particular application unusable (through a combination of factors accidentally producing white-on-white text or whatever).
I'm assuming that GTK will break API/ABI where it's necessary to enable future work, but will not break API/ABI just for the sake of it. I don't know what that means for themes: perhaps the fact that they touch every widget means they're likely to be affected by at least one breakage, but perhaps that's balanced out by the move from unique GTK'isms to something very close to standard W3C CSS.
One interesting side effect of being more willing to break API/ABI is that the GTK maintainers would be more free to expose APIs they aren't 100% sure about being able to retain for multiple years of development, such as the CSS "gadget" APIs that Matthias mentioned in his blog post - if I'm understanding it correctly, that would make it much easier for custom widgets outside GTK to be "first class citizens" with the same level of CSS support as the widgets in GTK. If an API turns out to have been a bad idea, it can be dropped immediately (during the SONAME-breaking development period), and won't show up in the next 2ish-year major release.