I have made this document for my own (future) use, but given the amount of internet searching I have done in order to find the info I thought I might just share it with the rest of the world.
I have a software, mdadm, RAID 1 mirror in my computer. It works fine, and once setup, I never have to think about it. But then I run out of space, hard disk prices go down, and I migrate the system to new larger disks. And never remember neither how I set it up nor how I migrated the last time. So this time around I decided to document it for future use.
This is how I have set up my computer:
Disks → Partitions → RAID mirror → Physical Volume → Volume Group → Logical Volume → File System
[Lower layer → Upper layer]
There are reasons why I have this setup. Using LVM makes the system agnostic of the underlying hardware, which is perfect when devices fails, are moved, etc. The system does not care, but keeps booting and running while waiting to be fixed. It is a home computer, so I really do not need a lot of different partitions for the various mount points. Keeping swap inside the mirror means there is not much need to think if a drive needs to be replaced; simply partition the drive and let mdadm do the rest/job. Done.
I have tested this setup with removed drives, trashed partition tables, power cut, etc., and it is really robust. I like that.
There are several ways to do this. I will list two of them. Both have their up- and downsides.
The idea here is to let mdadm do the job for us. That way, we do not need to worry about where the superblocks reside and not, and so forth.
Skip to the cookie recipe section below if you want to go straight down to business.
We know from the man page that mdadm mdadm will set up mirrors from partitions of different sizes, choosing the smallest one as the resulting mirror size. We also know that mdadm is able to grow an array if there is free space in the partitions the mirror uses. Then the workflow is:
After that, all the other layers above (physical volume, logical volumes, file system) can be resized one after another.
The coolest thing with this approach is that everything is done while the system is up an running normally!
/dev/sda | The first old disk |
/dev/sdb | The second old disk |
/dev/sdc | The first new (larger) disk |
/dev/sdd | The second new (larger) disk |
/dev/md0 | The RAID mirror |
vg00 | The volume group |
/dev/mapper/vg00-swap | The swap logical volume |
/dev/mapper/vg00-root | The root logical volume |
The idea here is to create an entirely new RAID mirror and then copy everything from the old one to it, finishing with running a bootloader installation to make it bootable.
Skip to the cookie recipe section below if you want to go straight down to business.
The benefits of this method is that the copying part is faster than the RAID sync, and that you get a newly created, unfragmented file system). The downside is that your system will be offline when you have created the new array and need to copy the contents from the old filesystem to the new one, you will need a rescue disk or live system and you need new names for your RAID array and volume group (and thus need to change /etc/fstab and /etc/lilo.conf in the new file system accordingly).
/dev/sda | The first old disk |
/dev/sdb | The second old disk |
/dev/sdc | The first new (larger) disk |
/dev/sdd | The second new (larger) disk |
/dev/md0 | The old RAID mirror |
/dev/md1 | The new RAID mirror |
vg00 | The old volume group |
vg01 | The new volume group |
/dev/mapper/vg00-swap | The old swap logical volume |
/dev/mapper/vg00-root | The old root logical volume |
/dev/mapper/vg01-swap | The new swap logical volume |
/dev/mapper/vg01-root | The new root logical volume |
When I migrated from 1 TB disks to 2 TB disks, I ran into a strange problem that I had never seen before. Running LILO in order to install correctly on the new set would fail with a “device-mapper: Sector outside mapped device?” error. This was both when I had created a set with “modern” partitions starting at sector 2048 and using the old standard with partitions starting at sector 63. If you try to find info about that with Google (or even Bing...), you will see that there is not much to find but the LILO source code that produces the output.
I got the advice to try GRUB instead, but it would fail to install because /boot/grub/core.img was 771 bytes too big to fit before sector 63 (and at that point I did not feel very much like redoing it all again to get the partitions back to starting at sector 2048). Since MBR is quite stone age, I will use GPT in the future, but at this point I did not feel very much to fiddle around with that either. My assumption was that with cloned disks and the only difference being the enlarged partitions/RAID set/Volume Group/Logical Volume/File System, probably the large size would be the issue. Thinking that the swap volume, that I created first, would probably reside in the beginning of the disk, I shrunk the swap volume, created a new logical volume for boot in that space, rsync:ed the contents of /boot, mounted that new logical volume for boot at /boot (and remembered to update /etc/fstab), ran lilo, and voilà, it worked. (For now?)
Please feel free to e-mail me if you have positive comments or concrete suggestions of updates.