Wubi and Grub issues Part II

In case you missed all the excitement on ubuntuforums.org, a new bug (I mean grub update) has arisen affecting Wubi installs. This bug is the same one that is breaking Wubi upgrades from 10.04.1 to 10.10 and has been passed down to the 10.04 Lucid Lynx LTS now due to a totally unnecessary cosmetic grub update.

Any Wubi install that is on the same partition as Windows i.e. the C: 'drive' is impacted. Shockingly it appears the grub update wasn't even tested - a brand new install of 10.04.1 followed by the grub update renders it unbootable.

So what can I do to fix it?
Unfortunately the fix (Problem #2, Solution #1) is complex, especially for a new Ubuntu user. This fix has been developed by users on Ubuntuforums.org without the help of the developer. This isn't really surprising based on his comment here regarding the 10.10 upgrade issue:

[14:38] <jibel> cjwatson, did you saw my update on bug 653134  ?
[14:38] ...
[14:38] <ubottu> Launchpad bug 653134 in lupin (Ubuntu Maverick) "Can't boot Ubuntu after an upgrade from 10.04.1 to 10.10" [High,Triaged] https://launchpad.net/bugs/653134
[14:39] <cjwatson> jibel: I doubt there is anything else I can do now
[14:39] <cjwatson> loopback loop0 /ubuntu/disks/root.disk <- don't see how it could boot at all without that
[14:40] <cjwatson> lang=fr/lang=en> hmm
[14:41] <cjwatson> I guess I could try a non-English installation, since that sounds like a serious contributor to the problem
[14:41]  * cjwatson hates life
[14:41] <jibel> cjwatson, I must admit that I don't understand either, I commented the lines one by one until I've found the one that's blocking the boot.
[14:42] <cjwatson> wubi upgrades have never worked right before.  it would have been nice to say that they worked this time, but I have a hard time considering it a blocker
[14:43] <cjwatson> (this is not to say I don't appreciate efforts to get them to boot ...)
[14:43] <cjwatson> er, to work
[14:43] <cjwatson> wubi really needs a maintainer who isn't me.  Agostino has been mostly lacking in time
[14:44] <cjwatson> I'm only doing it by default

Could this be the final nail in the coffin for Wubi?
It's ironic that those careful users, who read the 10.10 release notes and stayed wisely on 10.04, but religiously install all Important updates can no longer boot. Even Wubi's most ardent supporters must find it hard to defend the product in the face of such unreliability.

All in all, Canonical should either relegate Wubi to the unsupported fringe, or else shake up their development staff: releasing production fixes without any testing!!!???? Hello!?

Resizing your WUBI install

When you first run WUBI it asks you how much space to allocate to it. It will default to a reasonable value based on the space available on the target partition, but you can override it if you wish. For many people this is the first look at Ubuntu and they accept the default.

Others select a windows partition (or 'drive') that has lots of space, believing that Ubuntu will be able to access this directly. It cannot. Once you've fixed the size of the install it creates a virtual disk (root.disk) that is the selected size, and will remain that way.

Of course you can store and access files on the Windows host partition, in the /host directory - but that's not always enough.

So, how to resize the virtual disk? Basically you create a new virtual disk in the size you want, and copy the old one to it. This doesn't allow you to 'add a couple of GB to your existing disk' - you need enough room for the entire new size (plus enough remaining space for Windows).

You can use this guide on ubuntuforums.org to do the resize.

My personal recommendation is not to resize, but instead to bite the bullet and migrate to a partition install. But if this is not possible, a resize is always an alternative, provided you have the space.

How to install Ubuntu using WUBI

Installing Ubuntu with WUBI is really easy. You download the wubi.exe installer from www.ubuntu.com/desktop/get-ubuntu/windows-installer and when prompted, save and run it. You need to be an administrator (on Vista/7 you can right click and select 'Run as administrator').

That's by far the easiest way, but it's not always ideal...

Download time

Wubi.exe is only 1.4MB to download, but it has to download the Ubuntu CD image which is 699MB, and it's not very efficient. You can save time by downloading the CD .iso image yourself. If you need to reinstall you'll save even more time.

Picking your release

You don't get to pick which version you want. Currently the version supplied is 10.04.1 Lucid Lynx (the first update of the current long term release version). Maybe you want to have the 10.10 Maverick Meerkat instead, or maybe you want to try an older release. If you download the Ubuntu image yourself at the desired release, and burn it to CD, then you will automatically get the version you wanted.

The 10.04.1 looping download bug

The windows installer is very particular about which release it will install. If you download the 10.04.1 version of wubi.exe, it can only install a 10.04.1 version of Ubuntu (or variant) with it. The problem is that not all variant's of Ubuntu have a 10.04.1 release. The windows installer knows this and proceeds to download the 10.04 image of the variant. Then it checks it and discovers - gasp - that it's not 10.04.1. Then it loops back and proceeds to download the image again. If you happened to have left your computer during the download you could come back to discover it's downloaded and rejected the same image 3 or 4 times. This is another one of those simple bugs that could probably be fixed in a day, but 4 months following the release of 10.04.1 it's still sitting untouched.
PS avoid the Netbook-edition and Xubuntu with the 10.04.1 version of wubi.exe

What other options are there?

You can download the image you want using bittorrent. Then download and place wubi.exe in the same folder as the .iso file and run it from there. The windows installer will find the image and use it (provided it meets the release requirement and the MD5 checksum is a match). This is also useful if you want the 32-bit version of Ubuntu, but your computer is 64-bit capable (Wubi.exe will always download the 64-bit version in that case).

Better still, download and burn the .iso image to a CD. This is very useful as not only will the version of wubi.exe always match, but now you can boot Ubuntu from the CD and try it without changing anything on your computer. This will tell you whether Ubuntu is compatible with your hardware. The other important thing is you now have a rescue disk to bail you out if and when something goes wrong. (e.g. some of those grub prompts I mentioned in my other post).

Final notes...

The windows installer is written in python and - whether an issue with python or wubi I can't say - it complains about certain types of hardware. E.g. certain multimedia card readers, iPhones, and certain other peripherals. It's best to remove everything you don't need before running wubi.exe, otherwise you'll get irritating popups about disk errors that you have to continually click to get through (it's not an infinite loop, but it appears that way). Also, if you're running Windows 7, don't try and install Ubuntu to a dynamic drive that is not also listed in the partition table.

Pick the size of your Wubi install carefully. It's not easy to change the size of the virtual disk once you've created it. When you run out of space, there are options to resize, but at that point - it's probably better to migrate to a direct partition install.

Remember that the Wubi Ubuntu is running off a virtual disk that is a single file. It's much easier for a single file to be corrupted. Avoid hard shutdowns even if it looks like the computer is hanging (use Alt+SysReq R-S-U-B instead), and take care to backup your data.

WUBI and GRUB Issues

Edit Dec 16: If you're currently having problems with Wubi booting or get a grub rescue prompt when booting, see this post: Wubi Mega-help-thread

Since version 9.10, Ubuntu has used Grub2 as the default bootloader. (I'll refer to Grub2 as Grub, as the previous version is now referred to as grub-legacy. )

Grub stands for GRand Unified Bootloader. And it's so grand that everyone is talking about it. You shouldn't notice a good bootloader or need to know it's name, but once you've turned on your computer and seen the Grub prompt screen, or the optimistically named Grub rescue prompt (which means you're totally messed), it's hard not to take notice.

Let's look at some of the significant Grub problems with Wubi:

Karmic Koala 9.10
Problem: Select Ubuntu after running system updates and the result is a grub prompt.
Reason: The wubildr (Wubi bootloader) file can't address files that are 4GB or more from the start of the partition.
Fix: Users must manually download a new copy of wubildr.
Note: This bug is part of the release, so every new 9.10 install will be affected by this.

Lucid Lynx 10.04
Problem: Many users render their computer unbootable, destroy their Windows partition bootsectors. In some cases Windows can not be repaired and requires a reinstall. Some users inadvertently end up losing all their data through bad repair attempts.
Reason: Running the upgrade to 10.04 or even the standard grub updates asks you to select a device to install grub. It confusingly lists all drives and partitions in a user unfriendly way (/dev/sda, /dev/sda1, /dev/sda2 etc.) and then cautions to 'select all drives if uncertain'.
Fix: If you got the right help, reinstalling the windows bootloader and restoring the partition bootsector was all you needed to do. This worked for most people, but as I mentioned, some were not so fortunate.
Note: the developers didn't even respond to the bug report for about 2 months following the release, despite hundreds of affected users (documented cases, but probably thousands were affected). Even non-wubi dual boot installs were affected.

Lucid Lynx 10.04.1
Problem 1: Many users render their computer unbootable.
Reason 1: A fix for the 10.04 grub issue has been released. For 10.04.1 you cannot overwrite your drive bootloader with a grub update if you have a wubi install. Problem is, the means of identifying a wubi install is buggy - only those who installed to drive 'C:' (or their main windows partition) are safe.
Fix 1: Reinstall windows bootloader.
Note 1: The bug that prevents grub from identifying a wubi install is still present in release 10.10.

Problem 2: After selecting Ubuntu, you see errors listed and either hang, reboot, or end at a grub prompt.
Reason 2: Grub now has the ability to update the wubildr on the fly. Since grub is a rapidly evolving product, it needs to update the wubildr to be compatible with all the changes. Somewhere between updating the modules, the grub menu and the wubildr file things go bad. Exact cause unknown. Only affects installs capable of updating the wubildr (on the same partition as Windows)
Fix 2: Loop mounting the root.disk from a 'Live CD' and manually editing the /boot/grub/grub.cfg file (just leave the parts you need to boot i.e. anything on or after the first line starting with "menuentry". For some reason, after booting and running "sudo update-grub" the problem disappears. [Edit: a subsequent grub update in November, is not permanently fixed by running update-grub. Manual clean up of the /boot/grub directory is required - see later posts for info]


Maverick Meerkat 10.10
Problem: After upgrading from 10.04.1 to 10.10, your computer either hangs, reboots, or drops you at a grub prompt.
Reason: Same reason as for problem 2 on 10.04.1 - for installs to the same partition as Windows, the wubildr file is updated and corrupted.
Fix: Copying the backup wubildr from the c:\ubuntu\winboot directory over c:\wubildr seems to fix it. Or loop mounting the root.disk and manually editing the /boot/grub/grub.cfg file (just leave the parts you need to boot i.e. anything on or after the first  starting with "menuentry".
Note: There is no permanent known fix to this and the bug report has not been addressed by the developers since before the release. (Unlike for the 10.04 bugs, more proactive testing by community members identified the problems, early - but unfortunately once 10.10 was released the developers seem to have lost interest and moved on to Natty Narwhal 11.04).

So what can you do to avoid these problems?
Don't update grub-pc, grub-common or lupin-support packages. Don't play with grub unnecessarily. Don't upgrade to a new release without taking a full backup and having a repair CD handy.
The ultimate prevention is to migrate your Wubi install to a normal partition install.

Ubuntu with WUBI

If you're wondering about trying Ubuntu, but all you've ever known is Windows (and it came preinstalled) - then likely the thought of partitioning and setting up a dual boot is not pleasant.

Fortunately for you, it isn't necessary either... it is possible to install Ubuntu from within Windows and experience it as a dual-boot without any partitioning. If you don't like it, you can simply go to Add/Remove programs and uninstall it (like any other Windows application).

Q. Does it run Ubuntu within windows?

No, only the WUBI installer/uninstaller runs within windows. Ubuntu runs as a proper dual-boot. This means it has full access to all your system resources and can run on as little as 256MB RAM (although I'd say a minimum of 512MB is probably a good idea).

Q. Is it slower than Ubuntu?

Yes, but not noticeably so. It's a lot faster (and simpler) than running Ubuntu in a virtual machine under Windows.

Q. How does it work?

In simple terms, WUBI creates a virtual disk that is simply a large file under the Windows file system. It then runs the standard Ubuntu installer, but instructs it to install to the virtual disk. This avoids the need for creating a separate partition.

It also adds a special form of the Grub bootloader into the Windows boot manager, which allows you to select either Windows or Ubuntu when you start up the machine.

Q. What's the down side?

WUBI is mostly used by newcomers to Ubuntu. That means it doesn't get as thorough testing coverage or support as regular installs. Many Ubuntu community members are dead-set against WUBI and can be very unhelpful e.g. you regularly see advice to uninstall as soon as you ask for help, which makes them feel better but doesn't address any of the issues.

I'll address the sorts of problems that are WUBI-specific in other posts, but what you need to know is that once a WUBI install is booting (once you are past the GRUB menu), any issue you see is likely an issue between Ubuntu and your hardware and is unrelated to WUBI.

Q. How do I safeguard my WUBI install

Do not ever install the Grub bootloader to any devices. Grub updates have a very misleading interface and although you need it for WUBI, you do not need to install it's bootloader yourself ever. When you see a screen popup for you to select where to install it, leave all boxes unchecked.

Try and avoid hard shutdowns on your computer. Since WUBI places the entire Ubuntu install on a virtual disk (a single file) it can corrupt if a write is pending and the computer is turned off. If something hangs, then try anything and everything to regain control. The last resort is the following: hold down Alt+SysReq and then slowly press the following keys R-S-U-B.

As for all computer use, backup your data. If you want to be cautious you can back up the entire virtual disk C:\ubuntu\disks\root.disk (change drive letter if required).

Read the release notes before trying to upgrade to a new Ubuntu release. Take special precautions to backup all c:\wubi* files and the c:\ubuntu directory before upgrading.

How to migrate a WUBI install to partition

If you've used WUBI for a while, you've likely spent a lot of time customizing it: playing with settings, installing and configuring programs, and saving data. If you planned ahead you made notes along the way and could reinstall fairly easily, but chances are it'd take a lot of effort to do this again.

When you refer to the Wubi Guide, it points you towards LVPM to do the migration, but LVPM hasn't been updated since Ubuntu 8.04, and doesn't work out of the box without some tweaking. The Wubi Guide's own script is in a similar state.

Fortunately it is still possible to migrate, and you can find a script and a step-by-step guide(if you want to do it manually) on ubuntuforums.org. *

Before you migrate you'll need to partition your drive. That's something you'll have to do beforehand, but it's fairly straightforward on Windows Vista or 7 - use the built in Disk Utility. For XP, you can use Gparted that is part of the Ubuntu 'Live CD'.

* This can be used to migrate Ubuntu versions 9.10 and later. If you installed on an earlier version of Ubuntu (even if you later upgraded) you have a different boot loader and the guide does not apply.