Oneric Ocelot 11.10 released

Update: Usually not much changes on the day of a release, certainly not on Wubi. However, it looks like a number of last minute changes were made, generating new revisions of wubi.exe, from revision 241 (which is the version on the current Oneiric CD ISOs) to revision 251, before it looks like they settled on release 245 (which is the version on the main Ubuntu.com download site). If you're wondering what that was all about, you're not alone, but from what I gather something to do with bugs installing Xubuntu/Kubuntu with Wubi. At any rate, my tests has been mostly on revision 241, and only on Ubuntu. If I become aware of outstanding issues, I'll create new posts to address that rather than redo my testing on multiple revisions of Wubi.

Now that 11.10 has been released there are a few things that can be documented:

Preinstalled images (1-stage install)
In the absence of an Ubuntu CD or downloaded CD image (ISO), wubi.exe will download a 'pre-installed' compressed image instead of the full desktop CD ISO. This saves about 200MB on the download and, more importantly, skips entirely the '2nd stage' install that Wubi users used to go through - a considerable time saving. Users will have to download and install some language packs separately though.

Differences
1. With the conventional 2-stage wubi install, you'd get a menu prior to the 2nd stage install - if you hit ESC during the 5 second countdown. From that menu you could select from 4 boot options, and also provide your own manual kernel boot option overrides. With the preinstalled image, this no longer works. If you need to supply kernel boot option overrides these will have to be done before rebooting by editing the file \ubuntu\install\wubildr-disk.cfg in Windows, specifically this line:

linux /vmlinuz root=UUID=$diskuuid loop=/ubuntu/disks/root.disk preseed/file=/ubuntu/install/preseed.cfg wubi-diskimage ro quiet splash <add options here>

Please note that if you don't have this file, you're likely doing the normal 2-stage install. For editing I recommend WordPad or write.exe as the line endings are linux style.

2. If you have Windows 7 and Microsoft Office's clickonce virtual drive Q: then you'll get a "Permission Denied" exception at the end of Wubi's windows (stage 1) installation. It's okay - it looks fatal - but it's not. Just reboot and it will work. (Note there are other reasons for getting Permission Denied - if you're unsure, review the log file wubi-11.10-rev24x.log in the %temp% directory).

3. There are still a couple of bugs on the preinstalled image. The /etc/fstab is not setup correctly, only containing a reference to the swap file, and even that is not valid (at least based on the traditional Wubi swap entry). The file system is also supposed to be ext4, but is currently still ext3. Fortunately, these 'preinstalled' images could probably be updated after release (although I have no idea if they'll be frozen like other release elements). 

FYI the current /etc/fstab looks like this:

# UNCONFIGURED FSTAB FOR BASE SYSTEM
/host/ubuntu/disks/swap.disk    none    swap    sw      0       0

It should look like this (once the image is corrected to ext4 you can adjust accordingly):

proc   /proc proc   nodev,noexec,nosuid   0 0
/host/ubuntu/disks/root.disk / ext3 loop,errors=remount-ro 0 1
/host/ubuntu/disks/swap.disk none swap loop,sw 0 0

Note: the bad /etc/fstab doesn't appear to cause any problems, at least, not that I've noticed in limited testing.

Normal 2-stage install
The conventional Wubi 2-stage install is still supported for offline installs, or when you run it from a CD. When wubi.exe detects a desktop CD ISO it will bypass the pre-installed image download (even if it rejects the local ISO it will download a new ISO).

Note: that the issue #2 for preinstalled images - the Permission Denied on the Q: drive - does not occur in this case.


Other notes
1. On Vista/Win7, when you reboot after the Windows intall stage is completed, it will boot into Ubuntu directly without presentation of the Windows Boot Manager (one-time only). This will happen even if you suffer from the bug where the 'Time to display operating systems' is set at zero (the symptom will be that it boots one time directly and after that, just straight into Windows) - for everyone else they'll see the windows boot manager at the next boot and thereafter (with Windows as the default).

2. Although Lubuntu is now an officially supported release 'flavour', and wubi.exe is included on the Lubuntu CD ISO - wubi will not support a Lubuntu install (although it is possible with some manual workarounds).

3. Ubuntu 11.10 introduces a new display manager (lightdm) which requires an update to the Wubi migration and resize scripts referred to in the Wubi Guide. These will be updated shortly following the release.

If you have any questions, go to https://answers.launchpad.net/wubi/+addquestion (launchpad account required) or http://ubuntuforums.org/newthread.php?do=newthread&f=331 (remember to prefix title with [wubi]). Also http://askubuntu.com/questions/tagged/wubi is a great resource - check if your question has already been answered first before posting on askubuntu.com.

Oneiric beta and Wubi

I've been running Oneiric on Wubi since early on in development, and it has worked very well. But a few issues have come up with Beta1 and 2:
1. Offline installs are no longer working, and
2. The new Wubi pre-installed image doesn't work properly

Offline installs
When you're testing in development, it's best to use something like zsync to keep your Ubuntu installation CD ISO up to date - it changes pretty much daily as developers release package update frequently - and if you don't use zsync you have to download the whole desktop CD image (700MB) when zsync might only need 100MB. By placing the ISO and Wubi.exe in the same folder, you can install it easily and quickly. Even if the ISO isn't the latest, disconnecting from the internet will stop Wubi trying to download the latest.

But this is no longer working since Beta1. Wubi.exe no longer looks in the current directory - it just downloads the new pre-installed image (more about that later). There is a workaround... copy the ISO to the root of a partition e.g. E:\ and Wubi thinks it's a CD and will use it.

New install method
So what is this new install method? Wubi.exe used to download the desktop CD ISO when it was run standalone (and didn't find a suitable image in the current directory or other drives). Then it would preseed Ubiquity with some special instructions to format and install to the virtual disk. This means Wubi had a two stage install - first Windows, and second, the regular Ubuntu install (ubiquity).

New with Oneiric Beta, when Wubi.exe is run standalone, it downloads a pre-installed Ubuntu image and expands that on the virtual disk. So this removes the need for the second, Ubuntu install stage. It just boots the image it downloads, and the preseed file just contains the user info, which is used to setup the users account details.

So that's great - it's quicker to boot. But the problem is it's not quite working. The swap file is no longer setup, there is no /etc/fstab so performance can be sluggish. Another bug that didn't setup the grub.cfg on the new image has been fixed so hopefully these other issues will be fixed soon as well.
Note that the preinstalled image uses the ext3 file system (Wubi has used ext4 since release 9.10) - I'm not sure what the reason for this is.

Oneiric Alpha3 Wubi

I've been running Oneiric with Wubi since early on in the development cycle - but it was an upgraded Natty. I finally got around to doing some installation testing on Oneiric - a week or so after the Alpha3 was released.

The first thing I tried was just running off the latest daily ISO. I downloaded the latest wubi.exe from here and let it download the ISO. It took a long time, probably because the bittorrent download isn't very fast when there aren't many seeds. I guess it's faster to download through the browser. At any rate, a couple of hours later, I did the install (wubi.exe and the oneiric iso in the same folder) and let it run. Once I rebooted to complete the install, it failed. Apparently the kernel and initrd on the ISO were bad as it couldn't mount my windows partition. There was a helpful message to run chkdsk /r and reboot twice into Windows, but that was wishful thinking. This is part of alpha testing - probably if I tried on Monday it would work. I'll find out later and create a bug if it doesn't.

So I decided to drop back to the alpha3. The only issue with this is that the ISO is oversized so it won't fit on a CD (and I downloaded it on another machine). So rather than burn a DVD I reused an Ubuntu USB stick I had sitting around. Note, I just copied the ISO to the root of the USB - installing Wubi from an Ubuntu USB doesn't work (in most cases), but it's okay if you just copy the ISO to the root. Note: if you're trying to install an older version of the ISO like this you have to disable your internet connection or wubi.exe will try to download the latest available ISO.

Anyway - the install went fine. The only issue was that it took a long time 'detecting file systems'. I had a peek in the logs and it seemed that it was trying to mount the root.disk, but whether this was the cause of the delay I cannot say. I'll dig into that in more depth later.

Then I ran all updates (250MB since Alpha 3 was released) - because there are some cool changes to Unity that I read about and I wanted to check out.

I've been running Oneiric on my another computer (i3, 4GB RAM) and it's been great. For the alpha test I used an older machine (core 2 duo, 1GB RAM) and it's very sluggish. But it works okay, and Unity 3D is working on the X1300 ATI card.

So in summary, Oneiric Alpha 3 worked fine with Wubi. The more testing the better - so try it out, file some bugs, help make it better for everybody. If you have questions, post them at https://answers.launchpad.net/wubi/+addquestion

Missing the root.disk

I wrote earlier about the Mystery of the disappearing root.disk. I've seen a number of additional cases since that post - and not all have been catastrophic, so thought I'd discuss what the steps are to attempt recovery.

My belief is that this problem is largely the result of manually shutting down your computer when the Wubi install is hanging. But there have been cases where corruption has occurred without manual poweroffs and I've noticed that some files I delete from within Wubi are 'recovered' later by Windows.

But it is important for users to resist the urge to 'hit the restart/power button' when the Wubi install is hanging. For any OS this is a bad idea and can lead to problems. For Wubi installs it can be fatal. Instead refer to the Wubi Guide for other options.

The first thing you'll notice may be Windows running an automatic CHKDSK or when you try to boot Ubuntu you end up at a GRUB> prompt.
Then when you boot Windows and look in the C:\ubuntu\disks directory you'll notice the root.disk is missing. In some cases, the \ubuntu\disks directory is missing completely or is corrupted.

Running chkdsk
Depending on the problem, Windows may have run an automatic chkdsk or you may need to run it manually. It's not always necessary, but it will never hurt, so the first thing is to run it.
Go to My Computer on XP or Computer on Vista/7, right click the drive you installed Ubuntu on e.g. right click on C:, select Properties, select the Tools tab, then under Error-checking click Check now. Select to Automatically fix file system errors or Scan for and attempt recovery of bad sectors (I usually don't use this, but if you hard drive has problems it's a good idea). When the drive you installed on is C: the computer will tell you it has to schedule the scan for the next time you start your computer. Reboot to complete.

Locate recovered files/directories
The first thing to do is to look for the \found.000 folder on the drive in question i.e. C:\found.0000. This is hidden by default and (on my Windows 7 install) I also had to tell Windows not to "Hide protected OS files" just to see it. You may also have to recover from an administrator command prompt on Win7 (see below).

So now you look for your root.disk (or other .disk files) and copy them back to the \ubuntu\disks folder. If the entire \ubuntu\disks folder is missing, you'll likely find a dir0000.chk directory and within that the root.disk, swap.disk and empty \boot\grub folders. Copy this back to \ubuntu renaming the directory to disks.

If you're missing the root.disk but there is no file of that name, it may have been renamed chk0000.chk. Rename this to root.disk and copy back to \ubuntu\disks.

If the corruption was minor, then likely everything will work fine. If the corruption is major Windows may not even recover the root.disk at all.

Win7/Vista command line instructions
Hit the START key, enter CMD, then look above and right click on CMD.exe and select "Run as Administrator", as shown below. From Windows 8, type "CMD" on the Metro page, right click, and then look on the bottom for "Run as administrator.

Check for the hidden recovery directories: dir /a:h

Check each \found.??? directory:

C:\>cd \found.000
C:\found.000>dir
 Volume in drive C is OS
 Volume Serial Number is B4B7-99A8

 Directory of C:\found.000

19/07/2011  02:02 PM    15,000,000,000 file0000.chk
               1 File(s) 15,000,000,000 bytes
               0 Dir(s)  222,258,069,504 bytes free

C:\found.000>move file0000.chk \ubuntu\disks\root.disk
        1 file(s) moved.

Or if the whole \ubuntu\disks folder is missing:

C:\>cd \found.000

C:\found.000>dir
 Volume in drive C is OS
 Volume Serial Number is B4B7-99A8

 Directory of C:\found.000

19/10/2012  04:51 PM    <DIR>          .
19/10/2012  04:51 PM    <DIR>          ..
19/07/2011  02:02 PM    <DIR>          dir0000.chk
               0 File(s)              0 bytes
               3 Dir(s)  222,258,069,504 bytes free

C:\found.000>dir dir0000.chk
 Volume in drive C is OS
 Volume Serial Number is B4B7-99A8

 Directory of C:\found.000\dir0000.chk

19/10/2012  04:51 PM    <DIR>          .
19/10/2012  04:51 PM    <DIR>          ..
24/02/2012  12:22 AM    <DIR>          boot
06/11/2012  09:28 AM    13,000,000,000 root.disk
15/11/2011  09:28 PM       268,435,456 swap.disk
               2 File(s) 13,268,435,456 bytes
               3 Dir(s)  127,904,968,704 bytes free

C:\>move dir0000.chk \ubuntu\disks
        1 dir(s) moved.

I hope this helps you to recover your files. Remember to backup important data on your Wubi install. There's no reason to install important personal files on a root.disk - you can access them easily on the /host partition.

Updated 2012-11-06: added some more screenshots, enhanced DOS commands

Wubi boot issues a thing of the past

With the release of the third update of Lucid Lynx (10.04.3) last week, this finally puts to bed the problems with Wubi and Grub2 (Ubuntu's bootloader) that resulted in many Wubi boot failures. The issue that allowed the user to overwrite the Windows bootloader is also taken care of.

These issues were fixed prior to the release of Natty Narwhal earlier this year, however, the patches to the active stable releases (Lucid and Maverick) were only released as an Update in June, and finally in 10.04.3 last week.

This was certainly a big problem that has plagued Wubi users for over a year; longer if you include the problem in 9.10 Karmic Koala. It kept many on ubuntuforums.org busy and led to the successful Wubi megathread (thanks to Rubi1200 for that).

I have been quite critical of Canonical and the developers on this blog for not addressing this sooner (this problem was probably the driving force behind the blog in the first place), however, I think it's true that my focus is quite narrow and the developers have many issues to deal with, and Wubi wasn't always on the top of that list.

It appears that now there is a strong development effort to improve Wubi for 11.10 Oneiric Ocelot - which I'll write about later. So far, I've been running a development Oneiric Ocelot Wubi install from early on in the development cycle without any issues.

One thing seems certain - Wubi is popular as ever with new users and a great tool to introduce Windows users to Ubuntu.

How to migrate - in pictures

Update: this post is now out of date. Please refer to this instead: How to migrate updated for release 12.04

Installing Wubi - common problems

Since the release of Natty Narwhal 11.04 a few days ago, I've been monitoring various support forums looking for issues. The good news is that I'm not seeing anything new and exciting, just the standard problems users have with installs. So that's good, but I realised that now is probably a good time to catalogue some of these common problems - and their solutions (if they exist).

Preamble
Wubi installs involve two distinct processes. First the Windows install where you select the user name and password, the target drive, and the size of the install. Wubi then finds a local desktop CD image or downloads one, creates virtual disks, modifies the Windows boot manager, sets up a special file with installation instructions and tells you to reboot.
The second part is when you reboot and select Ubuntu from the Windows boot manager. This boots a linux kernel (through grub4dos and grub2), and then runs the standard Ubuntu installer (Ubiquity), which uses the special installation instructions created in Windows - so that no user interaction is required  (usually).

Important info
If you want an Ubuntu CD/USB: go to http://www.ubuntu.com/download/ubuntu/download (step 2, click on medium, and then SHOW ME HOW)
You can run wubi from the CD or standalone. If you run it standalone you can download the desktop CD ISO yourself and place it in the same folder as wubi.exe and it will use it (must be the same release).
You cannot install from an alternate ISO or a DVD ISO (burning a CD ISO to DVD is okay). Installing from a USB will fail if the USB partition is > 890MB (and it will copy the entire partition to your hard drive before failing - and that could be an 8GB file, so be careful).

When Wubi fails in Windows you should check the log file or post it on a support forum for help. To find the wubi log file, go to the %temp% folder (enter %temp% in your Windows explorer address bar) and open the file wubi-nn.nn-revnnn.log (n's represent release and wubi.exe revision)

Windows issues

• Failure to download the metalink. Either you don't have internet access or you are installing 11.04 and you select Ubuntu-netbook. I mentioned the 11.04 Netbook problem in my last post, so nothing further to add there. 
• You select to install to a drive that is actually a Dynamic disk. These are Windows specific and it won't let you 'boot from them' - and Ubuntu cannot see or boot from them either (unless they also happen to reside in the MBR partition table - in which case you run the risk of destroying data on dynamic drives that do not).  Solution: DO NOT INSTALL on dynamic drives. Either get rid of them or find an external drive.
• You get a firewall notification asking whether to allow "pyrun.exe". This is the python executable that is actually running Wubi - and it is uses a bittorrent downloader to retrieve the desktop CD image. You need to give it access or the install will fail. Alternatively download the desktop CD ISO and place it in the same folder as wubi.exe (and ALSO remove the bad CD or wubi.exe will find it and fail again).
• The CD was not created correctly (a bad burn). If you look in the wubi log it will tell you the md5sum failed or some other error (Errno13, 22). Solution: create a new CD or run wubi.exe standalone.
• It's downloading the wrong version 64bit. It's not actually the wrong version. Wubi defaults to 64bit when the computer is capable (and some users do not know that their computers are 64bit). If you want 32bit then download the 32bit version and a) place it in the same folder as wubi.exe, b) burn it to CD or c) specify the command line option --32bit when running wubi.exe
• IOError: [Errno 22] or [Errno 13]. This error can be associated with a bad burn, bad image, or even an optical drive incompatibility. Most likely a bad burn. You can either boot from the CD and get it to check itself, or remove the CD, and run wubi standalone (with or without the downloaded desktop CD ISO in the same folder). If you are not running from a CD, Errno13 can be associated with a permissions error (run as an administrator) or a corrupted file (run chkdsk).
• Pyrun.exe says "No disk". This looks like a tight loop but it's not. Usually it's caused by a disk 'drive' assigned to an empty drive like a multimedia card reader. Either remove the cardreader or other peripheral or click cancel many times to get through it. To kill the install process open up task manager and kill pyrun.exe.

Please note - if you have an Ubuntu CD (matching the release and flavour of wubi.exe) in a disk drive, Wubi.exe (even if run standalone) will find it and use it. If the CD has a bad checksum or bad burn, it will cause the installation to fail - Wubi.exe might go through the motions of downloading a new CD, but often the first error will be fatal. 

When you reboot from Windows to complete the installation, and select Ubuntu the errors you encounter are related to the Windows boot manager, grub4dos, linux/hardware compatibilites, and the Ubuntu installer (Ubiquity).

Ubuntu/Ubiquity issues

• If you find it boots straight back into Windows, then either Wubi failed to add an entry to the Windows Boot Manager, or the Windows boot manager Timeout is set to 0. In both cases, let it boot, right click on (My) Computer, Properties, Advanced, Startup & Recovery settings... and check the Timeout - make it 15 or more. Check the drop down box to see if the Ubuntu entry is there. NEVER set Ubuntu as default and the Timeout to 0 or 1 or Windows won't boot.
• "Try hd(0,0): xxxx: No wubildr". This is not an error. Ignore it unless it accompanies "grldr not found". See one of my previous posts on this.
• Reboots back to the Windows boot loader or grub> prompt. See the Wubi megathread, problem #2 (This won't happen on an initial install)
• Grub rescue> prompt. See the Wubi megathread, Problem #1. (This won't happen on an initial install).
• If you never get to see the installer, then you are having a problem booting the linux kernel. There could be a number of reasons... graphics card, other device incompatibilities. The best thing to do is search the support forums (e.g. ubuntuforums.org) for your particular computer brand/model and the specific release - often someone else has figured out what you'll need and published it. See here for how to supply boot workarounds.
• You installer starts, but you get the message "no root file system is defined". This is often caused by Ubiquity failing to read the target partition. This message can also occur on a normal install,for a different reason (user error) but with Wubi there is no user interaction to select the target root partition. Some common causes are: you have some leftover GPT partition table data when you're using an MBR partition table; you have some fakeraid metadata but are not using raid, or you're using an unsupported Raid. In all cases, it's best to run the bootinfoscript and create a request for help on the forums to find a solution.
• Other problems. If the installer halts for any reason, you have to collect logs otherwise no one will be able to do anything about it. Drop to terminal (CTRL+ALT+F2) and collect the logs (to \ubuntu\installation\logs.zip): 

sudo sh /host/ubuntu/install/custom-installation/hooks/failure-command.sh

No more Netbook-edition on Natty Narwhall 11.04

Good news for Wubi users - Natty Narwhal 11.04 is out and it is probably the safest Wubi install for some time, mainly due to all the grub2 fixes that came out recently.

There's one gotcha that was missed during development and this is that Wubi, when run standalone, will still give you the choice between Ubuntu and Ubuntu-netbook. However if you choose Ubuntu-netbook you'll get an error message that it couldn't download the metalink.

What's the reason? The netbook-edition in Maverick was basically the same as 32-bit Ubuntu except it included the Unity interface. With 11.04 the newly rebuilt Unity interface is now standard on regular Ubuntu, so there's no need for a separate Netbook-edition.

Wubi Natty will be ready for Beta2

Wubi has seen some major focus from Canonical's developers (Colin Watson) in the past week or so, finding and fixing major bugs including long-standing Grub-related bugs affecting 10.04 Lucid Lynx and 10.10 Maverick Meerkat (these releases will see the fixes a little later). But for 11.04 Natty Narwhal, Wubi is looking stable now for Beta2 - or if you're using the current-daily CD images.

This means that both new 11.04 installs as well as upgrades from 10.10 to 11.04 are now working!

PS There's just one innocuous message you'll see before the grub menu is displayed, regarding the prefix not being set.

Natty is natty

In my previous post I mentioned that Natty is now working with Wubi (on the current daily-live image), and also that the new Unity desktop is now working on my old Dell with an ATI X1300 Radeon (whereas with 10.10 netbook edition and early Natty Alphas, Unity did not work at all).

Since then I've reinstalled a number of times and I'm coming to like Natty and Unity. It feels - well - "natty". Not that that's a word I'd use frequently and I consider it only marginally better than the upcoming Oneiric Ocelot (that must be some good weed over at Canonical).

The only Wubi issues I've found are a couple of boot problems with the latest version of Grub. The first was fixed very quickly, and the second only seems to affect my installs when they're not on the main Windows partition. I've worked around this by manually booting and then editing grub.cfg by hand.  I believe that it's basically the same grub rebooting issue from 10.04 and upgrades to 10.10, but this time there is no workaround - other than changing the  00_header script in /etc/grub.d/ but hopefully this will be resolved before release.

The more people that test and report bugs, the better. So if you decide to try Natty on Wubi, please report any bugs you find in launchpad.net and make it better for newcomers to Ubuntu.

Wubi finally working in Natty Narwhal

I've just tested the latest Wubi.exe available here. It took a while to download the latest daily-live Desktop CD image (in the end I did that myself from here). But after that installing Wubi was painless. Apart from a benign grub2 message, the Ubiquity installer was impressive - looking ready for release - and seemed faster than prior releases, as did booting up for the first time.

Surprisingly Unity even worked on my computer - I'm not totally sold on it and there were some rendering issues, but at least now I have the opportunity to review it, instead of staring at a blank screen.

So - in summary - the installation of Wubi Natty Narwhal worked flawlessly. As always, alpha testing has risks, so take necessary precautions.

Wubi pulled from Natty Alpha3

For those hoping to test Natty Narwhal 11.04 using Wubi, it will not be fixed in time for Alpha3 (the bug report is available here). The new target for a fix is Beta1.

If you read my previous posts, you will know that you could install Alpha1 with Wubi (with some workarounds), but with Alpha2 a new ubiquity bug prevented this from working. I haven't tested this lately, so I don't know whether ubiquity has been fixed. But it's safe to assume that the only option now for Wubi Natty testing is to install Maverick 10.10 and then upgrade.

It shouldn't really be a surprise. Wubi is intended for the newcomer to Ubuntu so having a working Wubi option early in development is not a top priority. Although I am puzzled what the major difficulty is. Also, the fact that there are so many major changes to grub and ubiquity between Maverick and Natty should be sobering for those that like to upgrade to each new release (these don't seem to be very stable products). When you also factor in the major changes to the front-end (Unity) and it seems to me that the Ubuntu developers are being stretched pretty thin. At the same time there've been some pretty major bugs in Lucid that have gone unpatched for over a year.

Wubi itself is due for an upgrade. With a 2007 version of grub4dos that hangs up on ext4 partitions, and some pretty bad usability bugs (e.g. the python code that chokes on multimedia card readers) and confusing (meaningless) error messages... someone needs to step up.

New release of Wubi migration script

I've released a new version of the Wubi migration script (wubi-move-2.0.sh). Here are some of the new features:

• You can migrate either Wubi or a normal install. It detects the type and handles accordingly. This is useful if you want to move your regular Ubuntu install, keep a working backup, or do some experimentation without jeopardizing anything. Also for Wubi users who migrate and then need to re-jig their partitions, it makes it easier to move the install (without worrying about UUID changes).
• It now supports grub-legacy. This was one area that was lacking - I don't know how many people have Wubi with grub-legacy, but there have been a few requests. So the migration supports releases with grub-legacy: 8.04 to 9.04 (as well as those that have been upgraded to 9.10 and later). The caveat is that the script replaces grub-legacy with Grub2 on the migrated install - so you probably don't want to be doing this prior to release 9.10 (I don't know how robust Grub2 is on 8.04.4, for example).
• It also supports migration from a root.disk file with the --root-disk= option. I had already come up with a manual solution for this before and it didn't seem too big a deal to add in to the script. So if you have a good root.disk, but you've lost the ability to boot into it, or you want to migrate to another machine, you can now boot a live CD and migrate from it. The root.disk must be a working, fully-contained Wubi install i.e. not have separate virtual disks for /boot, /usr or /home. (This rules out grub-legacy where /boot is always on the /host Windows partition.)
• Finally there are some minor tweaks. The new option --shared-swap will bypass the 'mkswap' command to preserve the UUID on the swap partition. Useful if you have multiple installs and you want to share the existing swap partition.
• I also split the output from --help into two separate options: --help and --notes to suppress the flood of output.
• Other notable changes - better validation, better error handling, for instance if there is an error in the chroot, the script attempts to cleanly unmount from the chroot before exiting. 

Testing all the possible scenarios and releases is a challenge. I've done a number of cases from releases 8.04.4 to 10.10 (except 8.10) including Wubi and non-Wubi, with Grub2 and Grub-legacy. But as always community feedback is always welcome.

Anticipating Natty Alpha 2 with Wubi

So, it's Alpha 2 week and what better time to check out Wubi again. But the news isn't good. In fact, things have gone from bad to worse. Before you could install Natty Wubi by replacing the buggy wubildr with a working Maverick copy to kick-start the installation. Now it just results in a loop in the installler (ubiquity) saying "No root filesystem has been defined".

So much for that... Let's try the upgrade option instead.

And the silver lining is... that the upgrade from Wubi Maverick to Natty was very smooth. No errors. No strange popups. No Afghanistan keyboard. In fact, the only thing I noticed was an "error: file not found" as grub booted into kernel 2.6.38-1.

So upgrading a Maverick install is still the preferred way to go if you want to test Natty on Wubi.

PS I didn't expect the fresh Natty Wubi install to work because the Natty wubi.exe hadn't been updated (since early December). And the new installer bug isn't too surprising either - it's Alpha after all and you need to expect these things. However, I do think it's fair to expect that sometime between December 2nd and February 2nd that the developer would have done one or two unit tests on Wubi.exe and discovered the problem  (or at least noticed the bug report).

To be honest, I'd prefer they stopped messing with Wubi Natty and fixed the far more important Lucid Grub issues... but I know that's not likely to happen.

Just today there was someone on Ubuntuforums.org who had their computer completely wiped (Factory restored) by some lousy 'tech support' due to the Wubi grub rescue issue. These are real, live, normal people trying out Ubuntu and it'd be nice if the devs took that a little more seriously.

The mystery of the disappearing root.disk

In the past month or so I've noticed a small but noticeable number of users losing their root.disk files (or the entire \ubuntu\disks\ directory). The reason for this is unclear and, so far, the recovery rate is 0%.

In one case, booting from a live CD and running "ls /host/ubuntu/disks gave the following:
# cd /media/win/ubuntu/disks
# ls -l
# d??????? ? ? ? ? ? disks

It's hard to know from the other side of the internet what attempts have been made to recover. But suggestions to run chkdsk don't seem to fix the problem. It's also impossible to know whether this is a new Wubi problem or whether this is a regular issue that has always affected Wubi. Time will tell.

What can you do in this case? The best insurance is a regular backup of your root.disk file, or the important contents contained within.

On that note, I also saw a thread where a user tried to reinstall Wubi to fix a problem. Reinstalling Wubi will always remove the current install, delete the root.disk, and destroy any existing Ubuntu data/settings.

"Just uninstall"
Unfortunately I also see many community members advising Wubi users to uninstall because they assume the problem is Wubi-related. Often this is not the case, but rather it is a normal Ubuntu problem that can be solved (and would still have to be solved after replacing Wubi with a normal install). It is important to let these well-meaning members know that this advice could destroy the user's important data. It's not hard to add a cautionary warning, and perhaps suggest to the user how to recover data from a root.disk from Windows (e.g. http://ext2read.blogspot.com/) before uninstalling.

A final comment... in my opinion, reinstalling an OS is not a solution to a problem - it's the last resort when there is no solution. All too often I see users quickly reinstalling at the first sign of a problem e.g. when their master boot record MBR is overwritten by Grub2. This is the simplest thing to fix and doesn't damage installed OS's or their data, so if they could just resist the urge to give up and reinstall it would save them a lot of trouble.

And remember - you can rid yourself of Wubi, but not lose any of the look and feel, settings or data on your current Wubi install, simply by migrating. There is a new migration script coming soon that supports grub-legacy (Wubi installed prior to release 9.10 and later upgraded) as well as migrations from a live CD/USB simply by pointing at a root.disk file.

WUBILDR, WUBILDR.MBR and GRLDR

It's fairly common to see messages such as Try (hd0,0) : FAT16 when booting Wubi. Usually it's innocuous, but sometimes booting fails. You might see it stuck on Try (hd0,0) : Ext2  


Rarely you'll see something like:
Try (hd0,0) : FAT16: No WUBILDR
Try (hd0,1) : NTSF5: No wubildr
Try (hd0,2) : FAT32: No WUBILDR
Try (hd0,3) : invalid or null
Try (fd,0) : invalid or null
ERROR: Cannot find GRLDR in all devices: Press ctrl+alt+delete to restart


So what does this all mean, and what is GRLDR? 


Wubi uses GRUB4DOS combined with Grub2 to boot Ubuntu. GRUB4DOS has two parts: grldr.mbr and grldr. Wubi has renamed these to wubildr.mbr and wubildr (you are free to rename them as long as you insert the new name of grldr in the .mbr file). 


The wubildr.mbr file corresponds to a grub legacy bootloader in the Master Boot Record. It is limited to scanning all partitions to find the rest of itself - the part that doesn't fit in the tiny MBR. These are the messages you see as it scans all partitions - and since it is based on grub legacy it outputs partitions with a zero index (unlike Grub2 which still uses zero based indexing on the drives, but not on partitions) i.e. hd0,0 in wubildr.mbr corresponds to hd0,1 in Grub2.


No matter where wubildr.mbr is, when it is called, it starts scanning the root on ALL partitions in the order dictated by BIOS to find the wubildr file. If it doesn't find it, you get an error that it can't find GRLDR, not WUBILDR. The reason it says GRLDR is likely a hardcoded message - for Wubi it's always WUBILDR that it can't find.


So what are the reasons for failure to find WUBILDR? 
1. The rather obvious, but unlikely - there is no WUBILDR 
2. Excessive fragmentation apparently
3. The wubildr file is > 137GB from the start of the partition
4. You have an ext4 partition that is read prior to the one containing WUBILDR.

WUBILDR.MBR cannot read ext4 partitions. The GRUB4DOS code it uses is from 2007 and it simply hangs when it encounters one of these. If you have an ext4 partition higher up in the BIOS order than your Windows partition, you're out of luck.

Some interesting points: when you install Wubi it installs WUBILDR.MBR and WUBILDR to the windows drive C:. But it also installs it to other partitions, like recovery partitions. If these happen to be physically before the windows partition, WUBILDR.MBR will find this WUBILDR and never use the one on your C: partition (it uses the first one it finds). And this is interesting as Grub2 updates the WUBILDR on your /host partition - not always the one used. Maybe this is a clue as to why Grub2 updates no longer boot on some installs.

Natty 11.04 Alpha 1 Upgrade with Wubi - Latest

I ran the Natty upgrade again today from my backed up maverick root.disk using the daily Alternate CD (January 12*). There are some interesting changes. For one, lots more errors during the upgrade including one saying the whole upgrade failed. But it didn't.

And a nice change is that as I logged in - forgetting to switch from Unity to the Classic desktop - there was a popup informing me that my hardware did not support unity and, would I like to switch to classic? Great! It didn't work, but a good idea all the same - and I assume it will work soon.

So after re-creating the 'logout launcher' I escaped Unity and switched manually to Classic Desktop and then it logged in fine (apart from having to reload all the crashing panel objects.)

Here are some pics from the upgrade process. The following popup appeared at least 3 times (and might have something to do with the fact that Natty has my keyboard set up as USA/Afghanistan now) ...

This one looked serious:

This one sounded like it was toast:

Despite the serious message - the upgrade worked, Natty booted first time... and nothing too exciting has happened since.

* I ran the upgrade from the daily image of the Alternate CD on January 10 and January 12 successfully. The images shown are from Jan 12; there were no material differences that I've noticed between the two. When I attempted to do it on Jan 11 it wouldn't allow the upgrade to begin due to some package with an incorrect version. The message is, if you find that the upgrade fails one day, just wait and try the following day (this is normal for alpha testing since the repositories are updated frequently and sometimes packages get out of synch). Also, be careful if you are offered a Partial Upgrade when running the Update Manager in Natty. Sometimes this is required but often it's an out of synch set of updates so it's best to be sure - use apt-get update and dist-upgrade to figure out what will be removed and decide whether it's an error or required.

Wubi and hibernation

Wubi doesn't support hibernation... out of the box. But as Agostino Russo (all hail the creator) points out this is only because Wubi uses swap files, not partitions (if you could partition, why use Wubi?)

So yes it's true you can hibernate a Wubi install. And if you're just using it to test, you may well already have a swap partition lying around, so why not use it? Especially with 10.04 and 10.10 that seem to be a bit heavier on memory use than 9.10.

To set up hibernation on Wubi is the same as on a regular install. Just take the UUID of the swap partition and make sure it's in /etc/fstab and /etc/initramfs-tools/conf.d/resume. Then run "update-initramfs -u" to update the initial ram disk to point to it and look for it to resume from hibernation.
For example, assuming your swap is /dev/sda7:
$ sudo blkid /dev/sda7
/dev/sda7: UUID="xxxxxx" TYPE="swap"

$ cat /etc/fstab | grep swap
UUID=xxxxxx none swap sw 0 0

$ cat /etc/initramfs-tools/conf.d/resume
RESUME=UUID=xxxxxx

# if that's all good
$ sudo update-initramfs -u

Hint: if you are already using the Swap partition elsewhere don't run mkswap again or it will change the UUID.

Note: if you don't have a swap partition lying around and want to create one, make sure that it's big enough - to be safe I'd say the size of your RAM plus 1GB for overflow. Then run "sudo mkswap /dev/sda#" substituting # for your new swap partition and continue with the instructions above.
But if you're going to start creating partitions, I'd just take it one step further and migrate the Wubi install.

Warning on modifying Wubi Boot menus

When you install Ubuntu with Wubi, there really should be a release note popup - and it should contain some warnings:
1. Make sure you backup
2. Don't mess with the boot menus
3. Don't update packages grub-pc and grub-common

Well what can you say about point 1: Obvious. Point 3: covered ad nauseum already. But let's look at Point 2.

For some reason, many people are irritated about there being 2 boot menus. And they decide to remove one of them. And as I'll explain, that's not a good idea.

The first thing they try is to remove Grub. But grub is a little too smart for that - it prevents itself from being hidden if it detects multiple OS's. So you're out of luck unless you happened to find Ulrich Meierfrankenfeld's nice workaround.

So the next thing they try is to mess with the Windows Boot Manager. They have already noticed that the Grub menu already has an entry for Windows, so - they think - why not just set Ubuntu as the default in the Windows Boot Manager, and then set the Timeout to zero.

What they fail to realize is that the Windows option in Grub, does indeed boot Windows, however, Windows boots using it's Boot Manager - and they've just set that to boot Ubuntu automatically without a delay or prompt. Bye bye Windows.

So what can you do?
Apparently holding down the F8 key doesn't work as it's supposed to...

So if you're on XP, you can manually edit the boot.ini file from within Ubuntu to change the timeout - just be VERY careful - if you add the wrong line endings (i.e. carriage return/newline) then you'll be able to boot nothing - just get a HAL.DLL error.

If you're on Vista/7 I think you're out of luck. You will need a Windows repair disk and use the command line bcdedit to modify the timeout.

Final note, don't install burg on Wubi. It tends to install itself in the drive Master Boot Record and this stops your computer from booting.

More detailed Wubi grub analysis

Problem
A fresh Wubi 10.04.1 install will fail to boot after updating package grub-pc. When you select Ubuntu, the computer reboots, hangs or drops you to a grub prompt. This is nothing new - many people have been affected by this bug.

Analysis
A fresh Wubi install has two files in /boot/grub: grubenv and grub.cfg. That is all. When you update grub-pc, the script grub-install runs and this copies all the grub modules etc. into /boot/grub. This results in a modified grub.cfg as some of the scripts directly reference objects in /boot/grub, and the result is a Wubi that cannot boot.

For a fresh 10.04.1 install, the differences in grub.cfg before and after updating grub-pc are:
31a32,54
> insmod ntfs
> set root='(hd0,2)'
> search --no-floppy --fs-uuid --set 80e4e195e4e18daa
> loopback loop0 /ubuntu/disks/root.disk
> set root=(loop0)
> if loadfont /usr/share/grub/unicode.pf2 ; then
> set gfxmode=640x480
> insmod gfxterm
> insmod vbe
> if terminal_output gfxterm ; then true ; else
> # For backward compatibility with versions of terminal.mod that don't
> # understand terminal_output
> terminal gfxterm
> fi
> fi
> insmod ntfs
> set root='(hd0,2)'
> search --no-floppy --fs-uuid --set 80e4e195e4e18daa
> loopback loop0 /ubuntu/disks/root.disk
> set root=(loop0)
> set locale_dir=($root)/boot/grub/locale
> set lang=en
> insmod gettext

By resetting /boot/grub back to normal, the grub.cfg that is generated boots fine:

sudo mv /boot/grub /boot/grubold # backup /boot/grub
sudo mkdir /boot/grub # make a new /boot/grub

sudo cp /boot/grubold/grubenv /boot/grub # copy only required file

sudo update-grub # regenerate grub.cfg

That's 10.04.1...

With 10.10, there are some script changes in Grub. These result in a new function in grub.cfg: 

function load_video {

}

This is empty by default on Wubi installs. Actually it's empty on my normal Maverick install too. But the same fix on 10.10 for Wubi fixes boot problems on Maverick - regardless of the empty function.

PS you don't have to fix 10.10 unless you upgraded to 10.10 from 10.04.1 or - if you want to break it - just reinstall the same version of grub-pc.  But almost guaranteed the next update to grub-pc on Maverick will break it unless a fix for Wubi is provided beforehand.

11.04  Natty Alpha

On Natty, the grub.cfg is syntax checked by 'update-grub' (actually grub-mkconfig) before the update is completed. It finds and rejects this empty load_video function, and results in the grub.cfg NOT being generated at all. All you get is grub.cfg.new.

This affects a brand new Natty Wubi install, so there is no grub.cfg at all. Applying the same fix as for 10.04.1 does not work, as update-grub never completes. You can rename grub.cfg.new to grub.cfg and it boots, but this is a manual step, so not a fix. In any case, it seems that Natty Wubi will boot with the full complement of Grub modules in /boot/grub. It spits out a bunch of errors, but completes. When these errors cause reboots and when they are harmless is unclear but I guess it has to do with the WUBILDR file.

In any case, Natty is early Alpha, so we should expect weirdness. But if you find yourself confused, you're not alone.

Summary

On supported releases 10.04.1 and 10.10, Wubi installs fail after updating package grub-pc. The current community-provided workaround fix does not work on Natty - this is only significant because fixes are usually applied to development releases before supported releases, and to get the fix into 10.04.1 we probably need a fix that will work on all.

Upgrade to Natty 11.04 Alpha 1 with Wubi

An alternative to installing Wubi 11.04 Natty Alpha 1 from scratch - which is a convoluted process at the moment... is to upgrade from a current Maverick install. This bypasses the bad wubildr and Ubiquity installer issues. It does take a little bit longer to get setup, but it's quicker to repeat the upgrade.

Warning: with Alpha testing, something that works one day, may break the next. Please don't attempt this unless you have a dedicated test machine. The following example installs Wubi to the same partition as Windows (C:) - there is a difference, see note at bottom.

Step 1 - Install Wubi 10.10 Maverick
I picked 6GB for my install size.
Run all available updates.
Boot back into Windows and backup the root.disk and c:\wubildr (for repeat testing)

Step 2 - Download the daily 11.04 Natty Alternate CD
Since you'll probably want to repeat the upgrade at some point, it's a good idea to use zsync to keep your .iso up to date. If this is a once off, you could skip this step and just run the update online, but it takes much longer, in my experience than just downloading the alternate CD.

To download the 32-bit alternate CD:

zsync http://cdimage.ubuntu.com/daily/current/natty-alternate-i386.iso.zsync

See this for more info on zsync: https://help.ubuntu.com/community/ZsyncCdImage

Step 3 - Upgrade to Natty
Boot into the Wubi maverick install and run the upgrade from the alternate CD:

sudo mount -o loop natty-alternate-i386.iso /mnt
sudo /mnt/cdromupgrade

When you start the upgrade it asks you whether to check the internet for the latest packages - it's not really necessary since you have the latest daily image - and there's less chance of catching a buggy update - plus it's much quicker running just off the alternate image.

During the upgrade you get prompted a few times, but nothing exciting happened. I noticed by watching the grub.cfg file get regenerated (at least) 3 times and seeing the exception output, that the /boot/grub folder remains empty until the final time, when grub-install runs. This updates the wubildr and also places all the grub modules in /boot/grub. At the end the grub.cfg is regenerated in the same way that breaks wubi installs on 10.04.1 and upgrades to 10.10.

Step 4 - Reboot into Natty
The Wubi install didn't shut down when I requested a restart - it needed a little push (ALT+SysRq R-S-U-B). On restart, as expected due to the grub2/Wubi problems already covered in other posts, Wubi doesn't boot (but I had to try). I ended up at a grub prompt (not a rescue prompt), which is easy to boot from. I then chose the Classic desktop to login to, and had to click on multiple "Reload" buttons to fix the broken panel objects.

Welcome to Natty!!!

Step 5 - A new Grub2 wrinkle
So the accepted fix with Wubi installs is to clean out the /boot/grub directory and regenerate grub.cfg. This does spit out one error (on 10.10) but it's not an issue - about a missing /boot/grub/video.lst. However, with Natty, now Grub checks the syntax of the grub.cfg and rejects it if it finds errors. So the empty load_video function causes a syntax error.

You can get around this by leaving video.lst in /boot/grub, but that just spits out errors and reboots your machine when trying to boot your install. So... instead you have to manually rename grub.cfg.new to grub.cfg. Alternatively you can patch /etc/grub.d/00_header. What a pain! There must be an easier way to do this!!!

The instructions to patch grub, and the new command to manually rename grub.cfg.new follow:

sudo mv /boot/grub /boot/grubold
sudo mkdir /boot/grub
sudo cp /boot/grub/grubenv /boot/grub
sudo update-grub
sudo mv /boot/grub/grub.cfg.new /boot/grub/grub.cfg

That's about it. Natty boots fine after that, apart from having to click RELOAD on all the panel objects each time. Don't forget to select the "Classic" desktop unless you have a decent test machine that can handle Unity.

PS another nice thing is the Firefox beta. It looks very similar to Chrome - which is good for Firefox.

NOTE: if you install Wubi to a partition other than Windows, there is no way to update the wubildr file that is required to boot the Wubi install. Likely the upgrade will fail as the grub.cfg file has changed radically (I haven't tested it yet, but it pays to be cautious) and my previous test showed that the Maverick wubildr did not work. There are ways to generate a new wubildr file if you need to, but that's not covered in this article.