Why are packages in the official Ubuntu repositories older than the latest (upstream) versions from Debian Sid, PPAs, the authors, etc.?
An Ubuntu release goes through several stages before it actually makes it to the public as a finished product:
Some time before Ubuntu launches a release it freezes its packages at a certain point.
Before a release is out but after the package freezing, work is done mostly to fix all the bugs and issues that there might be in those packages. New package versions are not imported into the repositories anymore after package or feature freezing.
Once the release happens additional changes to those packages only happen for bug fixing and security issues. There are no more upgrades done to the packages in the official repository even if new versions of the packages are released.
Some packages have an exception to this, such as web browsers (which need to be kept up to date all the time) or certain cases.
New version of packages are consistently being imported (from Debian) for the next release of Ubuntu, until the next freeze happens and the same process repeats itself.
As an example, you can have a look at the release schedule of 12.04.
You can see that even though 12.04 was released in April, in January 12 something called _Debian Import Freeze_ happened.
This is just the first of many freeze stages happening before the actual release and means that at that point the import of packages from Debian testing or unstable stops and work starts on them to customize and fix issues with them.
No upgrades are done after that point in a lot of packages and the version that package had at that point is the version present and maintained through a release's lifetime.
So even though there are higher versions of the same package in developers' PPAs or in Ubuntu+1 repositories those will only be included in the next release of Ubuntu.
This is done for stability, security and functionality. New bleeding packages being imported all the time to the main repository would mean issues and a lot more problems to be solved. A freeze in the packages version helps to sort that out and make Ubuntu safer and more stable for the end user.
A new version of Ubuntu is released every 6 months, so every 6 months new packages are prepared, tested, customized and released with a new version. Future versions of a packages can be installed in your system via a PPA or just by downloading it from a web site, but the version of the package in the official repository remains the same.
For more understanding and an interesting overview of what happened to Ubuntu from 10.04 until the launch of 12.04 have a look at ReleaseSchedule - LTS to LTS and Stable Release Updates page for a complete overview and explanation of an Ubuntu stable release.
Two reasons. The first is quite obvious: it requires a human to spend time updating the package when a new upstream comes out. The second is that if you are running a stable release as opposed to the current development version, packages are intentionally NOT updated willy-nilly to avoid breakage. See http://wiki.ubuntu.com/StableReleaseUpdates.
Packages are frozen for the release and not updated subsequently for a number of reasons. If new releases were brought in post-release, then the new version...
- might bring new bugs, thereby regressing functionality that was present at the time of the release
- needs manpower to package, test, and upload
- needs its own set of security updates
- would need updated translations for its UI
- would need updated documentation (and translations)
- makes technical support more challenging
- might annoy users who have gotten used to the features in the old version
- may require newer dependencies which could break other apps if they were changed in the repository
- may break other packages that depend on this one
- may break user scripts, templates, tools, etc. created for the old version
All that said, be aware that there are cases where Ubuntu does do full updates of software versions in the repository. Firefox for instance.
Also, there is a ubuntu-backports repository users can opt into which updates software packages that won't cause problems like those listed above. It's not enabled by default so users have to opt-in to it, which is done to eliminate the surprise of having your software change out from under you. Also, it's not heavily staffed and so I'm not sure how frequently packages actually get updates.
Further, the SRU team has recently updated polices a bit which hopefully will make it a bit more straightforward to get bugfix-only package updates in.
Normally the updates in released versions of Ubuntu are for security and bug fixes, examples of such bugs include:
Bugs which may, under realistic circumstances, directly cause a security vulnerability. These are done by the security team and are documented at SecurityTeam/UpdateProcedures.
Bugs which represent severe regressions from the previous release of Ubuntu. This includes packages which are totally unusable, like being uninstallable or crashing on startup.
Bugs which may, under realistic circumstances, directly cause a loss of user data Bugs which do not fit under above categories, but (1) have an obviously safe patch and (2) affect an application rather than critical infrastructure packages (like X.org or the kernel).
For Long Term Support releases we regularly want to enable new hardware. Such changes are appropriate provided that we can ensure to not affect upgrades on existing hardware. For example, modaliases of newly introduced drivers must not overlap with previously shipped drivers. -New versions of commercial software in the Canonical partner archive.
-FTBFS(Fails To Build From Source) can also be considered. Please note that in main the release process ensures that there are no binaries which are not built from a current source. Usually those bugs should only be SRUed in conjunction with another bug fix.
-For new upstream versions of packages which provide new features, but don't fix critical bugs, a backport should be requested instead.
Taken from the excellent wiki page StableReleaseUpdates.
I shall try to answer your questions based on my past experiences from ubuntu forums and ubuntu planet.
I'm guess I'm just wondering how the apt repositories get updated, and by whom.
The APT repos do get updated from the packaging team at Ubuntu. The packaging team gets all the upstream packages from developers who do an initial packaging testing and other things. Then the testing team does the final testing giving a go signal. But the packaging team and the testing teams are very cautious about the dependencies and its side affects to the stable system.
When there's a lag, is it because the developer hasn't pushed the most recent release onto the relevant server?
If you see the upstream changes, there are thousands of developers wanting to push their packages. But not all are succeeded into the main stream this is because for various reasons. Assume Gedit application, a 2.2 version is suited and works fine with Dbus 2.1 and Gtk 2.4, etc. Where as Gedit 2.4 version (very new) needs Gtk 2.5 and Dbus2.3 to work. Now testing and packaging team (release team also)does not accept this because changing an existing system having old dbus and gtk with the new one breaks every thing else. Hope you got the point of dependency hell.
Is there a lot more work for the developer in getting the release into a form the repository can use?
Not to upstream channel. But to the release channel yes :).
P.S: There could be a bit changes done to the process now in the canonical compared to whats explained above. But it more or less same.
The accepted answer in the link fossfreedom posted as a comment is very good.
In general, package versions released after the first part of the new release development process don't appear in the main repositories of that release so that a reliable Ubuntu version can be tested thoroughly.
You may find some packages are released to the backports repository if they are sucessfully incorporated in a future Ubuntu release and if developers believe it will also work with earlier ones. Backports can be activated and deactivated in the Software Center (Edit->Software Sources->Updates Tab->Unsupported Updates)