How to upgrade a Wubi install

Yes it's that time again. A new Ubuntu release is around the corner, and that means that many people will be upgrading. If you have a Wubi install and look for advice, you'll probably be sternly warned that "Wubi wasn't intended for long term use... blah blah yawn". That's all great, but you're using Wubi and want to upgrade, and the reality is many people will do it anyway.

Upgrades fail

Shocking, but true. Ubuntu upgrades do fail. Frequently. Many people swear by fresh installs. Personally, I've upgraded to every release since 9.10 and with Wubi testing since 8.04. And 9 times out of 10 they worked okay (for me). The upgrade to 10.04 was probably the worst as you had to Ctrl-C at some point during a loop etc.

But nonetheless, every install is different and many fail (Wubi and normal both). So what to do?

Backup

This is a no-brainer. It's so easy to write it and read it and agree with it. But the way that the Update Manager suddenly jumps up "A new release is available. Do you want to upgrade?" - it seems so easy to just click that button. Even without reading the release notes (which may help you but not in all cases).

So... DON'T DO IT. Not until you've got a disaster recovery plan in place (the work-speak way of saying you have a backup and know how to restore it).

You should probably already have all your important data backed up, stored on the /host or other normal partition (or synched to Ubuntu One). Because data on the virtual disk is at a higher risk of being lost.

Full Wubi backups

On Wubi, it's really simple to do make a fully bootable backup. If you installed on your C: 'drive', then boot Windows, and copy the entire C:\ubuntu\disks folder. It's going to be big, so make sure you have enough free space, or copy it to an external drive if you need to. If things go wrong during the upgrade, you just have to copy it back and you have a full Ubuntu restore.

Preparation for upgrade

Make sure you have enough free space on the root.disk. Many people underestimate how much space is needed (and apparently the Ubuntu upgrade process also underestimates it, or has in the past). Personally I would make sure there's 5GB free, maybe 3GB minimum (that's sucking it out of my thumb, but the message is don't try it with the bare minimum).  Note, you can also resize the root.disk if you need to. At the same time, process all updates to the current release before hand.

To check free space, run df -h from the terminal (Ctrl+Alt+T). You're looking for the amount of free space on /dev/loop0 (that's the 'loop-mounted' root.disk)

Browse the forums

Take the time to see how bad the upgrade is. Other people will be doing it and posting the results (if they are bad). Get a feel for what the problems are.

Alternate CD upgrade

I strongly recommend using the Alternate CD to upgrade. Why? Because when the new release comes, every man and his dog starts to upgrade, and the servers struggle. So a 1 hour download suddenly becomes 3 hours or longer. Or times out. And you're up at 2am watching it (because at some point you're going to have to click that darn "OK" button for some meaningless prompt, or else you'll have to do it when you wake up).

Whereas, if you download the Alternate CD ISO, using a bittorrent client, it will likely take 30 minutes. Then you upgrade offline (maybe not recommended by Canonical, because this stops you downloading the latest packages) and you've just saved yourself a couple of hours. 

There's no rush

It's not going to hurt you to wait a month to upgrade. At that point, it should be very clear what problems there are if any, and there may even be fixes. Also the servers will be quieter, so doing an online upgrade gets you the latest packages. You can still run the Alternate upgrade, but let it download the latest packages online (still quicker than downloading all the new packages).

Caveats

Normally by this time in the development cycle, I've run a few Wubi upgrades. So I'm usually speaking from a position of knowledge that the upgrade can work. This time, I've been unable to test due to one of my computers dying, and a bunch of other time consuming things. Maybe I'll get the time to do this still (9 days to go as at the time of writing, in which case I'll add a comment to indicate that).

Update: I ran two upgrades, 10.04 to 12.04 and 11.10 to 12.04. I ran both online (not with the alternate CD - it helps having a 50Mbps connection and minimal server traffic). I confirmed that 2 GB is too little space - it downloads about 1800 packages and 'unpacks' them - and this uses a lot of space (that is mostly freed up at the end). So be warned - the upgrade tool does not stop you running out of space.
The 11.10 upgrade was on a real, in use installation, and worked flawlessly. The 10.04 was on a brand new, bare-bones 10.04.4 install with updates applied, and gave a number of errors and some weirdness (empty message box with 2 unknown buttons) - but succeeded as well.

Summary
1. Backup (always prepare for the worst)
2. Minimum 3 GB free space (preferably more)
3. 1800 packages to download so consider using the Alternate CD to speed things up

Another update: if you have a custom graphics driver, or use ppas, then it seems to be recommended to remove these before upgrading, and then reinstall following the upgrade.

How to migrate - updated for release 12.04 Precise Pangolin

A new version of the Wubi migration script will be released for Ubuntu 12.04 Precise Pangolin. There have been some minor changes that are required for release 12.04, and a few features added.

The simple migration hasn't changed

It's still straightforward to do a migration, for example if the target partition is /dev/sda5 and the swap partition is /dev/sda6, you can migrate as follows:

sudo bash wubi-move-2.2.sh /dev/sda5 /dev/sda6

New features

These are some of the changes with version 2.2 of the script:

• Simplified interaction
• Improved validation and diagnostics
• Migrate to separate target partitions e.g. for / (root), /boot, /usr, and /home
• Run migration validation pre-check without making any changes
• Resume a migration that failed due to e.g. a corrupt file, without having to recopy everything
• Synchronize a migrated install (e.g. to keep a fully bootable backup)

More information

For full instructions run:

bash wubi-move-2.2.sh --help

How to download

As usual, you can download the script from the HOWTO: migrate wubi install to partition thread on ubuntuforums.org. Select the file wubi-move-2.2.tar.gz, download, then right-click, and Expand. This will create a new directory with 3 scripts: wubi-move-2.2.sh, check-source.sh and check-targets.sh.

How to migrate in pictures

How to migrate from a root.disk

In the following example, the partition containing the root.disk is labeled 'wubi install' so the mountpoint is: /media/wubi install (including the space). This needs to be 'escaped' with '\' when specifying the location as shown in the screenshots below. Note also that this example uses a swap partition that is already used by another install (so option --shared-swap is specified to prevent running mkswap):

How to migrate to separate partitions

In the following example, there is a separate partition for /boot and /home in addition to the standard target (/) and swap partition. In addition, this example uses the  --shared-swap and --no-bootloader option:

Some notes on --resume and --synch

The new options --resume and --synch are designed to save time, either when the migration terminates due to rsync copy errors, or a refreshed migration is desired. Normally the script will only proceed on an empty partition (to prevent loss of data), so these options are the only way to bypass this - and they make use of rsync's sychronization ability to only copy files that are missing/changed.

Since this does introduce a little risk, these options rely on a control file created the first time the migration is started. This ensures that the target partitions are the same as on the prior run. In other words, resubmit the migration command in the exact same way, but append --resume on the end.

The idea behind the --synch option is to keep a bootable backup e.g. on an external drive. This can be quickly syncronized prior to performing major updates (like an release upgrade). 

Warning - synchronization will remove anything added to the migrated install that's not on the source install (in other words, don't use this option if you are actively using the migrated install).

This option is probably more useful for non-Wubi installs, so use it after you've migrated from Wubi (the script works on normal installs too).

Wubi disabled on live CD?

I wasn't going to blog about this... but since it made OMG! Ubuntu as a sensational article, I'd be remiss not to mention it.

Yes, it was floated on the ubuntu-devel mailing list to disable the ability to install using Wubi on the Ubuntu desktop CD. This didn't receive much discussion and so it's unclear what decision was made (if any), but I guess we'll find out soon enough with just a handful of weeks to go before 12.04 Precise Pangolin is released.

Background
Wubi.exe is released as part of the desktop CD ISO. This is fixed at release. So whenever you download the ISO you are getting the exact same one that others got on the release date... e.g. 11.04 - it's been out for a year, lots of updates, but the ISO is the same. Once you install Ubuntu 11.04, you'll be downloading 300+MB of patches. But since Wubi.exe is on the ISO it's also fixed for the duration of each cycle.

Why is this a problem? Wubi.exe typically gets limited testing during the development cycle. It doesn't get much development either - as the basic function doesn't usually change that much (11.10 was an exception to this). Most of this development tends to be after everything else is frozen, so the testing that was done early in development isn't always relevant. I could come up with other reasons, but you get the point. You're saddled with whatever bugs there are (some pretty irritating ones) - and even though many people download wubi.exe standalone, it's still fixed in stone.

Conclusion
So, in theory this is a great idea. We wouldn't have to live with irritating bugs, and instead of all the volunteers who support Wubi on Ubuntuforums.org having to repeat the same response 500 times, they could spend less effort making sure that the bugs are reported clearly so that a speedy fix can result.

However
It's still kind've nice to hand over a CD to someone and let them try out Ubuntu using Wubi, without having to download anything. I don't know how often this happens, but I guess there will be some who miss it. The question is, do you have to completely disable the CD install method to allow a stable release update to the standalone wubi.exe? Why not support both options?

Wubi call for testing

We're about 7 weeks away from the next LTS release, 12.04 Precise Pangolin, and this is a critical phase to get all the Wubi bugs ironed out. The reason is simple: after the release Wubi.exe is not modified so we'll have to live with any bugs for at least 6 months.

Wubi development testing is typically under-represented by the Ubuntu community because it's mostly used by newcomers to Ubuntu (and many hardcore users don't have Windows). Wubi is the first impression of Ubuntu for many, and so it's important that this is a good experience. Since 11.04 was released, Wubi has been very stable, but there's always room for improvement and to get rid of any irritating bugs.

With that in mind there has been a call to testing on ubuntuforums.org. If you have the ability then please help out, and provide feedback in that thread and/or file bugs on https://bugs.launchpad.net/wubi/+filebug.

There are a number of ways to run Wubi tests:
1. Download wubi.exe and run it standalone (during dev you can get it from here)
2. Download the Desktop CD ISO, burn it to CD if it fits (or DVD), and run it from there.

Note: each time you install Wubi it uninstalls any existing Wubi install (so if you use Wubi normally you can't easily test it unless you are comfortable backing up your current install).

There are a few tricks you can use. E.g. running wubi.exe standalone will download the preinstalled images i386.tar.xz or amd64.tar.xz from here. That's a 500MB download each time you run it, so instead you can download it once, save it, and then point it at your local copy with a command line option. This saves a lot of time if you are repeatedly testing. e.g. for a 32 bit pc:
wubi.exe --dimagepath=C:\xxx\i386.tar.xz

You can also test the ISO install without burning a CD or DVD each time, by saving it in the same directory as wubi.exe (which you can get off the ISO or use the standalone download).

If you run wubi.exe standalone, but install something other than 'plain' Ubuntu it will always download the ISO instead of the disk image. You can also override this download by copying the ISO into the same directory as Wubi.exe, but note that since the ISO changes frequently in development, chances are it will try to download a new one anyway. To get around this (if you know the MD5SUM of your ISO is good,) you can disconnect from the Internet for the first part of the install.

Remember you can use Wubi to install:

• Ubuntu
• Kubuntu
• Xubuntu
• Mythbuntu
• Lubuntu
• Edubuntu

All of these need to be tested. For Edubuntu, you download the DVD ISO (there is no CD ISO), but for the others you need the Desktop CD ISO.

Pay attention to the latest wubi.exe revision .e.g as of the time of writing, the latest is wubi-r256.exe. This version won't install Edubuntu, but forthcoming ones will. And whenever a new revision is published it's important to retest to make sure the bug fixes work and make sure there are no regressions.

Good luck.

Wubi must overcome some hurdles for Edubuntu

From 12.04 you will be able to install Edubuntu with Wubi. There are a few hurdles to overcome. The first is that Edubuntu requires a DVD image to install, and Wubi has built-in checks to prevent this - the ISO must be < 900,000,000 bytes (or 858MB) .

These same checks are what prevents users from installing a normal desktop CD image from a USB stick. These days USB sticks are generally large - in fact the Ubuntu download site states that a minimum of 2GB free space is required when creating a bootable USB.
But the way Wubi works is to copy the entire USB partition as the 'ISO' during the installation, and following that it performs the 900,000,000 byte check, and fails.
There is another danger because the USB partition is copied without any space checks. So imagine if you have 10GB space left on your hard drive, Wubi proceeds to install, but then tries to copy a 16GB USB stick partition onto that same hard drive. (That can only end badly). And even if you have enough space - it's going to fail the size check anyway.

Why is this an issue? More and more users are using USB sticks as they are cheaper, readily available and much less hassle than CDs (bad burn anyone?). But even more importantly, there have been reports that the Ubuntu desktop CD ISO will be increased 750MB size from release 12.04 on. This rules out the use of standards CDs. You can still burn a CD ISO onto a DVD - that works because the ISO does not expand to the DVD size in the same way as with a USB partition.

Relevant bugs:
Edubuntu https://bugs.launchpad.net/bugs/913354
USB https://bugs.launchpad.net/bugs/461566

Wubi will support Lubuntu in release 12.04

Lubuntu
Coming with 12.04 precise pangolin, Wubi will support Lubuntu. Preliminary tests using the daily live CD and this wubi.exe are successful. Note you have to logon blind for now while bug 918401 is sorted out.

I found that Lubuntu ran very well on my junky old test laptop - while it was very sluggish with 11.10 Ubuntu. So Lubuntu might be a better option for older machines.

If you're going to be testing Wubi remember to use zsync to keep your test ISOs up to date. It's a great tool.

Other wishlist
It'd be nice if Wubi could install from a USB stick with 12.04 - since the desktop CD ISO is supposed to be around 750MB it won't fit on a CD so I expect many people will try with USB. Waiting on longstanding bug 461566.

Also, it'd be nice if some of the obscure error messages that have appeared since 11.10 could be replaced by human-understandable text. For example "WindowsBackend object has no attribute 'iso-path'". I suppose it's no better than 'Permission denied' being used indiscriminately, but I suppose we were used to that.

For the 'preinstalled image' install method, I'd like to see an easier way to override the kernel boot options for users with ATI or NVIDIA cards requiring 'nomodeset'.

From a personal point of view I'd like to have an installable Wubi migration gui app ready - but it's not looking like it will happen in time.

Oneric Ocelot 11.10 released

Update: Usually not much changes on the day of a release, certainly not on Wubi. However, it looks like a number of last minute changes were made, generating new revisions of wubi.exe, from revision 241 (which is the version on the current Oneiric CD ISOs) to revision 251, before it looks like they settled on release 245 (which is the version on the main Ubuntu.com download site). If you're wondering what that was all about, you're not alone, but from what I gather something to do with bugs installing Xubuntu/Kubuntu with Wubi. At any rate, my tests has been mostly on revision 241, and only on Ubuntu. If I become aware of outstanding issues, I'll create new posts to address that rather than redo my testing on multiple revisions of Wubi.

Now that 11.10 has been released there are a few things that can be documented:

Preinstalled images (1-stage install)
In the absence of an Ubuntu CD or downloaded CD image (ISO), wubi.exe will download a 'pre-installed' compressed image instead of the full desktop CD ISO. This saves about 200MB on the download and, more importantly, skips entirely the '2nd stage' install that Wubi users used to go through - a considerable time saving. Users will have to download and install some language packs separately though.

Differences
1. With the conventional 2-stage wubi install, you'd get a menu prior to the 2nd stage install - if you hit ESC during the 5 second countdown. From that menu you could select from 4 boot options, and also provide your own manual kernel boot option overrides. With the preinstalled image, this no longer works. If you need to supply kernel boot option overrides these will have to be done before rebooting by editing the file \ubuntu\install\wubildr-disk.cfg in Windows, specifically this line:

linux /vmlinuz root=UUID=$diskuuid loop=/ubuntu/disks/root.disk preseed/file=/ubuntu/install/preseed.cfg wubi-diskimage ro quiet splash <add options here>

Please note that if you don't have this file, you're likely doing the normal 2-stage install. For editing I recommend WordPad or write.exe as the line endings are linux style.

2. If you have Windows 7 and Microsoft Office's clickonce virtual drive Q: then you'll get a "Permission Denied" exception at the end of Wubi's windows (stage 1) installation. It's okay - it looks fatal - but it's not. Just reboot and it will work. (Note there are other reasons for getting Permission Denied - if you're unsure, review the log file wubi-11.10-rev24x.log in the %temp% directory).

3. There are still a couple of bugs on the preinstalled image. The /etc/fstab is not setup correctly, only containing a reference to the swap file, and even that is not valid (at least based on the traditional Wubi swap entry). The file system is also supposed to be ext4, but is currently still ext3. Fortunately, these 'preinstalled' images could probably be updated after release (although I have no idea if they'll be frozen like other release elements). 

FYI the current /etc/fstab looks like this:

# UNCONFIGURED FSTAB FOR BASE SYSTEM
/host/ubuntu/disks/swap.disk    none    swap    sw      0       0

It should look like this (once the image is corrected to ext4 you can adjust accordingly):

proc   /proc proc   nodev,noexec,nosuid   0 0
/host/ubuntu/disks/root.disk / ext3 loop,errors=remount-ro 0 1
/host/ubuntu/disks/swap.disk none swap loop,sw 0 0

Note: the bad /etc/fstab doesn't appear to cause any problems, at least, not that I've noticed in limited testing.

Normal 2-stage install
The conventional Wubi 2-stage install is still supported for offline installs, or when you run it from a CD. When wubi.exe detects a desktop CD ISO it will bypass the pre-installed image download (even if it rejects the local ISO it will download a new ISO).

Note: that the issue #2 for preinstalled images - the Permission Denied on the Q: drive - does not occur in this case.


Other notes
1. On Vista/Win7, when you reboot after the Windows intall stage is completed, it will boot into Ubuntu directly without presentation of the Windows Boot Manager (one-time only). This will happen even if you suffer from the bug where the 'Time to display operating systems' is set at zero (the symptom will be that it boots one time directly and after that, just straight into Windows) - for everyone else they'll see the windows boot manager at the next boot and thereafter (with Windows as the default).

2. Although Lubuntu is now an officially supported release 'flavour', and wubi.exe is included on the Lubuntu CD ISO - wubi will not support a Lubuntu install (although it is possible with some manual workarounds).

3. Ubuntu 11.10 introduces a new display manager (lightdm) which requires an update to the Wubi migration and resize scripts referred to in the Wubi Guide. These will be updated shortly following the release.

If you have any questions, go to https://answers.launchpad.net/wubi/+addquestion (launchpad account required) or http://ubuntuforums.org/newthread.php?do=newthread&f=331 (remember to prefix title with [wubi]). Also http://askubuntu.com/questions/tagged/wubi is a great resource - check if your question has already been answered first before posting on askubuntu.com.