Anatomy of an Ubuntu update

So you probably figure that any mainstream OS would do some pretty major testing before releasing "Recommended" updates (these updates are set by default to notify users and are recommended for download).

Well, after the most recent Wubi-killing grub update I decided to investigate how exactly the Canonical approval process works examining the bug report and browsing the IRC logs. This update breaks a new 10.04 Wubi install. The results are shocking:

In the following quotes, bug 671097 is one that developer Scott Moser (smoser) was working on - regarding running Ubuntu Server in the Cloud - and bug 581760 is a cosmetic Wubi fix.

16 November: Comment from bug 671097

Just mentioning here, that cjwatson said this upload will wait for bug 581760 getting tested and moved to -updates.

This prompts Scott Moser to try and get bug 581760 verified in order to unblock his own work, despite not having any experience with Wubi (or else he'd know it required Windows to test)

16November: #ubuntu-devel on IRC

[20:49] <smoser> cjwatson, is bug 581760 testable without a windows installation ? [21:52] <cjwatson> smoser: no

19November: #ubuntu-devel on IRC

[22:25] <smoser> cjwatson, i tried to verify the fix for bug 581760 but apparently missed something. see my comments there.

Finally, it appears that a single test case has succeeded.

19 November: Comment from bug 581760:

I tried above again, this time disconnecting the network cord. - Installed from the same media mentioned. - rebooted, hitting escape at the grub loader after windows loader and verified it was 1.98-1ubuntu5 - let the install run to completion - reboot, verify still at 1.98-1ubuntu5 - did not add -proposed, 'apt-get install grub-pc' and saw confusing prompt (installing 1ubuntu7) Then, I uninstalled wubi and repeated above except for this time *did* add -proposed. I did *not* see the confusing prompt going from 1.98-1ubuntu5 to 1.98-1ubuntu8, verified that on reboot 1.98-1ubuntu8 was installed, and that reboot succeeded.

The developer does not seem overly confident about the single test case, but reports it as verified.

20November: #ubuntu-devel on IRC

[01:26] <smoser> cjwatson, so, if you see this, i've now (i think) sufficiently verified bug 581760 [01:28] <smoser> now we can get on to important bugs like bug 671097

The (apparent) lead developer figures he can release the fix, despite the deficient testing.

20 November: Comment from bug 581760

Sounds good to me. Thanks!

[Fix released on 20 November]

Summary

It appears that a developer unfamiliar with Wubi seems to stumble to a single successful test case. This is then considered sufficient to release the patch, which, when applied, breaks any default 10.04.1 Ubuntu Wubi install.

It appears that:

• Ubuntu maintenance on live releases are done without any formal test phase. Testing of patches are done on an ad-hoc basis and no testing plan is followed
• No independent QA testing or review/sign-off step is performed
• No facilities exist to remove a patch that is clearly fatally flawed, neither does there appear to be any interest from the developers in doing so, or at even, releasing a follow up fix.