Natty Narwhal 11.04 Alpha 1 with WUBI

So the adventurous often want to try out development releases of Ubuntu and what better way to test than with Wubi - in theory. In practice though, Wubi isn't the main focus and usually only gets considered closer to the release.

But curiosity killed the cat... so here goes:

1. Windows install
I use zsync to keep a current copy of the daily live image of the Natty desktop-i386.iso. Since the .iso change so frequently it saves time and ensures you always have an up to date copy. I extract the wubi.exe from it and copy both the .iso and wubi.exe to the same folder in Windows. Then run wubi.exe.
The Windows install GUI looks good - it's identical to previous releases except it shows 11.04 - and when finished, asks to reboot.

2. Ubiquity install

• Bad WUBILDR

So the first problem is the wubildr is bad. When you select Ubuntu from the Windows Boot Manager it hangs at Try (hd0,0) Wubildr. Replacing this with a wubildr from a Maverick install works fine and gets you to the second phase of the installation. (Note, in my case I had to replace wubildr on my first partition - a hidden rescue partition - not just the wubildr on my Windows partition. I did this via Ubuntu as it's not visible from Windows.

• Grub error during installation

After fixing the wubildr, the Ubiquity installer proceeds as normal with the slideshow. Until a popup appears entitled

Bootloader install failed.
Sorry, an error occurred and it was not possible to install the bootloader at the specified location. How would you like to proceed?

• Choose a different device to install the booloader on (empty drop down box)
• Continue without a bootloader (PICK THIS ONE)
• Cancel the installation

You'll be warned "You will need to manually install a bootloader in order to start Ubuntu" 

3. Booting the installed Natty
After installation, the first attempt to boot Natty drops you at a grub prompt.
GNU GRUB version 1.98+20100804-5ubuntu2.
I attempt to boot the install:
grub> configfile (hd0,msdos2)/ubuntu/winboot/wubildr.cfg

error: no such device: /ubuntu/install/boot/grub/grub.cfg
error: no such device: /grub.cfg
It is not possible to boot from the Ubuntu image.
Please verify blah blah blah and run chkdsk /r

So it turns out - after some investigation - that there is no /boot/grub/grub.cfg - when I skipped installing the bootloader it failed to create it. OK, no prob, just do a manual boot:

insmod ntfs
set root=(hd0,msdos2)
loopback loop0 /ubuntu/disks/root.disk
set root=(loop0)
linux /vmlinuz root=/dev/sda2 loop=/ubuntu/disks/root.disk ro
initrd /initrd.img
boot

Voila...
I then chose the classic desktop, as my last experience with the new Unity desktop didn't go so well:
Apart from a bunch of Error popup windows for panel objects that failed to load everything looks good.

First experience...
It looks pretty much the same as Maverick, but I notice there's now a "Root terminal" in Accessories. Love it! This is where I found the actual problem with the booting: there is no grub.cfg. I guess the grub error in ubiquity caused that part to be skipped. There's a grub.cfg.new so I just renamed that.

I decided to try out the new Unity desktop again, but it didn't work - I'm testing on my old junk PC with an ATI radeon 9000 card!

So now it's time to reboot and I discover that the Maverick wubildr is not compatible with the new Grub - every time I select Ubuntu the computer reboots. Back to the manual boot (I have another Ubuntu install so I can get to a grub prompt).

Rebuild wubildr...
Clearly Grub has changed radically - surprise surprise. It now refers to the drive as /dev/sda instead of hd0. So, that's probably why it's rebooting. Let's try regenerating the wubildr file. Oops logged in to the Unity desktop by accident... PS to logout I had to create a launcher on the desktop with command: /usr/bin/gnome-session-save --kill as I couldn't access any menus.
Then, to rebuild wubildr from root terminal: grub-install /dev/sda2 (don't try this on non wubi installs, also don't try it on wubi installs that aren't on the same partition as Windows's C: drive).

SUCCESS!
GNU GRUB 1.98
Ubuntu, Linux 2.6.37-10-generic
...

And it boots! Happy testing everybody*

(*And by everybody, I mean, those that consider their computers to be test machines - this is an early alpha release so you need to be prepared for disaster).

PS my windows is on /dev/sda2 - all places that refer to (hd0,2) or (hd0,msdos2) and /dev/sda2 are based on that.

Grub and real user experience

So I've focused mostly on highlighting technical issues from the Grub developer side that affect Wubi installs. Now I'd like to look at some real user experience.

First the meaning of Ubuntu... not the OS. One definition I like by Leymah Gbowee (according to Wikipedia.org) is "I am what I am because of who we all are". So when I see new users to Ubuntu staring into their broken computers - well, I think it reflects on everyone that is part of the Ubuntu (OS) community. 


Today there have been a flurry of requests for help with broken installs on ubuntuforums.org. One of them reads as follows:

Grub 2 Error. Deleted Windows XP. URGENT!!!

---

HELP PLEASE! URGENT!

I installed ubuntu 10.04 on my dad's laptop. I had partioned the HD, placing the ubuntu partition in FAT32 format. The install went through without a hitch. But then it began to update and halfway through, it asked me if i wanted to update the grub file. Knowing nothing about Linux, but thinking that if i didnt update it with the rest, it would be outdated and wouldnt work, i clicked yes and clicked the only HD i saw there as the option, as the forward button was unhighlighted when it was unselected. The rest of the install went through fine....Then the computer restarted.  And instead of getting to choose my OS's, i got a grub rescue screen. IDK what to do. My dad is ok with ubuntu being on it without XP. But the CD drive doesnt work and it cant boot from a USB so i cant reinstall. How do i fix this???

Please help.

Now, the fix as I mentioned elsewhere is simply to reinstall the bootloader. In this user's situation the complicating factor is that the CD drive is broken and the computer can't boot from USB. It's still not necessarily the end of the world - a trip to the computer store, or a visit to a friend who can extract the hard drive and fix it.


But it highlights something the developers don't seem to notice - the user experience: The panic; The fear; And hopefully the relief when it's resolved. But it's not really the ideal welcome to Ubuntu.

Wubi and grub problem update

It's been one month since the latest grub-pc update in version 10.04.1 Lucid Lynx, started breaking Wubi installs. And that's a good time to reassess the situation.

I'm not sure how I noticed it - maybe I was browsing Wubi bugs that day - but my quick alert to the issues didn't help. Again, I was hoping for a quick response, maybe the update being pulled - I don't know. But apparently launchpad doesn't have a recall mechanism. At the time I thought it would only affect users who didn't install on the same partition as Windows - the usual grub trick that gets users to install the grub bootloader to their drive masterbootrecord MBR caused by a bug in the 'lupin' package (see here, or here).
But quickly it became apparent that it also affected users who installed on the same partition - a problem we already knew about as it was discovered in the Maverick upgrade. But for 10.04.1 the problem is worse, and requires a manual loop mount of the virtual disk and an edit of the grub configuration file from a live CD.

So now we are on the Christmas break, and the developers get a well-needed rest from all the Natty development, but sadly noone has taken the time to patch lupin, for the many existing users on the supported releases. Still, the volunteers on ubuntuforums.org will soldier on, steering Wubi users to the solution... the Wubi megathread. It's interesting to note that since this thread was created 2 weeks ago, it's had over 2600 visits. And that's not including the number of people who had to be helped before it was created.

So... where are we at? There's not much really to be done to warn users unless they happen to post a support request for an unrelated Wubi question. I'm loathe to join the Wubi-bashing crowd  (although it's tempting): there's still a niche market for users that might cause more damage trying to do a proper partitioned Ubuntu installation. In fact, the new Ubiquity installer has it's own issues and has already caught out a few users and overwritten Windows and all other partitions - see bug 659106.
And in any case, despite all the Wubi-bashing, Wubi appears to be growing in popularity, fueled by its new prominent place on the ubuntu.com download page.

So... I guess we just wait it out, and hope that eventually the grub devs will grace us with their attention and provide a fix.

Canonical has a strange sense of priorities

So it's nearly a month since the latest Grub update in the current Long Term Release, Lucid Lynx 10.04, put the boots to Wubi installs. In addition to the usual alarming, but generally harmless, grub rescue> prompt problem,  which has been around since Lucid was first released in April, the newer failure to boot issue has been totally ignored by Canonical.
This boot failure problem started cropping up in July with grub errors about 'loadfont', but didn't seem to affect everyone. With the November 20 release it now appears to be much more widespread - a fresh install of 10.04.1 with the updates applied does not boot. (*Ubuntu must be installed to the same partition as Windows for this problem to occur).

The reason that it only affects Wubi installs on the same partition as Windows is that it is the grub-install command that is corrupting the /boot/grub directory, and when you update package grub-pc it runs grub-install to try to update the wubildr (Wubi bootloader) - believed by the grub devs to be a required action; BUT, Wubi installs on a different partition slip through the net as the buggy code fails to identify them as Wubi installs. This means that grub-install aborts and the wubildr is not updated - but ironically these still boot.
So the whole idea of updating wubildr and breaking Wubi installs in the process is pointless - why don't they just bypass the update altogether.

So... why haven't the devs rushed to fix this? Even the Ubuntu Release Manager Steve Langasek was notified of the 'critical' issue... but no response at all. The devs appear to be hard at work preparing for the next Ubuntu release (11.04 Natty Narwhal) - in the meantime, unsuspecting users who decide to try out Ubuntu are left to discover what a grub rescue prompt is, or why their new OS fails to boot.

So, in summary, Canonical doesn't seem to give a hoot about the current users who are breaking their computers trying out the 'user friendly' Wubi option. Are they just blindly churning out (buggy) release after (buggy) release as though this is a strategy for long-term success? Hard to say, but let's just say - I'm confused. If they don't want to support Wubi, why don't they just pull it from their main download site to protect the computers of unsuspecting users?

And thanks to everyone on Ubuntuforums.org who are working hard to help the affected users. It's great to have such strong community support, but if Canonical is relying on this for current release support it's pretty pathetic.

Where in the world is Ago

Agostino Russo, the person responsible for creating and maintaining Wubi, seems to have disappeared from sight. Since the run-up to the release of Ubuntu 10.04 (in April this year), he has been conspicuous by his absence.

From Ubuntu IRC logs, it seems that Colin Watson is now responsible for Wubi, but is either unable or unwilling to deal with seemingly minor, showstopper-type bugs that Wubi has suffered from lately. In fact, the bugs appear to be from Colin's own code changes in Grub2.

So what is Canonical's strategy on Wubi? If I were a conspiracy theorist I would suggest that the developers wish to have Wubi removed by making it so unstable as to be unusable. But I am not into conspiracy theories, and I instead believe that it's more likely due to lax maintenance/testing standards.

So that brings me to my point. Where is Ago? He's clearly the only one who has been able to keep Wubi going, and hopefully he can find the time to get things back on track.

Anatomy of an Ubuntu update

So you probably figure that any mainstream OS would do some pretty major testing before releasing "Recommended" updates (these updates are set by default to notify users and are recommended for download).

Well, after the most recent Wubi-killing grub update I decided to investigate how exactly the Canonical approval process works examining the bug report and browsing the IRC logs. This update breaks a new 10.04 Wubi install. The results are shocking:

In the following quotes, bug 671097 is one that developer Scott Moser (smoser) was working on - regarding running Ubuntu Server in the Cloud - and bug 581760 is a cosmetic Wubi fix.

16 November: Comment from bug 671097

Just mentioning here, that cjwatson said this upload will wait for bug 581760 getting tested and moved to -updates.

This prompts Scott Moser to try and get bug 581760 verified in order to unblock his own work, despite not having any experience with Wubi (or else he'd know it required Windows to test)

16November: #ubuntu-devel on IRC

[20:49] <smoser> cjwatson, is bug 581760 testable without a windows installation ? [21:52] <cjwatson> smoser: no

19November: #ubuntu-devel on IRC

[22:25] <smoser> cjwatson, i tried to verify the fix for bug 581760 but apparently missed something. see my comments there.

Finally, it appears that a single test case has succeeded.

19 November: Comment from bug 581760:

I tried above again, this time disconnecting the network cord. - Installed from the same media mentioned. - rebooted, hitting escape at the grub loader after windows loader and verified it was 1.98-1ubuntu5 - let the install run to completion - reboot, verify still at 1.98-1ubuntu5 - did not add -proposed, 'apt-get install grub-pc' and saw confusing prompt (installing 1ubuntu7) Then, I uninstalled wubi and repeated above except for this time *did* add -proposed. I did *not* see the confusing prompt going from 1.98-1ubuntu5 to 1.98-1ubuntu8, verified that on reboot 1.98-1ubuntu8 was installed, and that reboot succeeded.

The developer does not seem overly confident about the single test case, but reports it as verified.

20November: #ubuntu-devel on IRC

[01:26] <smoser> cjwatson, so, if you see this, i've now (i think) sufficiently verified bug 581760 [01:28] <smoser> now we can get on to important bugs like bug 671097

The (apparent) lead developer figures he can release the fix, despite the deficient testing.

20 November: Comment from bug 581760

Sounds good to me. Thanks!

[Fix released on 20 November]

Summary

It appears that a developer unfamiliar with Wubi seems to stumble to a single successful test case. This is then considered sufficient to release the patch, which, when applied, breaks any default 10.04.1 Ubuntu Wubi install.

It appears that:

• Ubuntu maintenance on live releases are done without any formal test phase. Testing of patches are done on an ad-hoc basis and no testing plan is followed
• No independent QA testing or review/sign-off step is performed
• No facilities exist to remove a patch that is clearly fatally flawed, neither does there appear to be any interest from the developers in doing so, or at even, releasing a follow up fix.

Wubi Mega-help-thread

There is now a one-stop Wubi 'megathread' on ubuntuforums.org to repair the grub issues Wubi users have been experiencing. Recommended reading if you are using Wubi, even if you haven't had any issues yet.

The thread is: http://ubuntuforums.org/showthread.php?t=1639198