This is based on some notes I collected after trying to find some documentation on starting the following setup manually, ie, when booted from some form of installation or rescue media (such as a Gentoo LiveCD). The more recent Linux kernels (both 2.4 and 2.6) have support for both software RAID and Logical Volume Management. Don't be fooled by the term "software RAID", as it works much better than the cheap "hardware" RAID controllers found on many recent mainboards; real hardware RAID controllers cost several hundreds to thousands of dollars...
So if you don't have a lot of money to spend on hardware, yet you have one of those cheap "hardware RAID" IDE controllers on your motherboard, or even better, a decent SCSI controller, or some of the newer SATA controllers, then you can use Linux kernel RAID in one of several configurations. Even without a hardware "RAID" controller, ie, if you only have the normal complement of IDE controllers you can still use RAID as long as you use one drive per controller (in this case, no slave drives allowed). SATA seems to be restricted to one device per controller, while SCSI can take up to 15 devices per (wide) SCSI channel (though you may max out the bandwidth of your SCSI bus, PCI bus, etc, if you use too many drives). With only one hard disk, you can still benefit from LVM (especially if you have a newer larger hard disk and you'd like to manage the space on it). LVM is great on top of RAID devices; just don't put a single LVM volume across multiple physical drives (ie, without RAID) since you'll lose all your data in the volume if just one drive fails (using one logical volume per physical disk is fine). Read the above HOWTOs if you're still wondering what this is all about.
Back to the point, both of the above rely on configuration files to some extent, as well as a special partition type and persistent superblock in the case of RAID (this is all documented in the HOWTOs, and also in additional docs such as the Gentoo Linux x86 with Software Raid and LVM2 Quick Install Guide).
Basically, the requirements include the proper kernel modules for raid and LVM2 (dm_mod for the latter), along with the right tools, which means a 2.6 kernel and both the mdadm and lvm2 packages in the case of Gentoo and most other recent distros. Both the modules and tools are available on the Gentoo LiveCD.
Note: one thing you should do, as soon as your raid/lvm setup is running, is create and backup some configuration files. Although you can usually get by without them, sometimes having the UUIDs and other information available can save your butt...
First, if you haven't already done so, create the mdadm.conf file and then copy it to one or more backup locations. The following command will show your raid device config, which you can also use to create the config file:
livecd ~ # mdadm --detail --scan
livecd ~ # mdadm --detail --scan > /etc/mdadm.conf
livecd ~ # cat /etc/mdadm.conf
ARRAY /dev/md4 level=raid1 num-devices=2 UUID=31c09f48:f8c019e2:601cfb52:058fda30
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=6f0a0c37:ab370856:c64eaf34:681027cb
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=1d100718:3438367a:6788bc93:36443a74
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=57f488f9:2d91902b:13ac3bd4:274ea500
MAILADDR root
MAILADDR root" directive so
that any errors will get mailed to an actual user; you can also add the DEVICE directive, but it's
not required (and could be problematic if device names change). This file can then be used
to start the array instead of manually assembling each device (as shown below).
The other important configuration file is lvm.conf; although the defaults should be
okay, if you restrict the device filter to a specific type (such as hd* or sd*) the lvm commands will
refuse to see an existing volume on any other device type. So be very careful and make sure the
"filter" setting is doing what you want. LVM2 will also automatically create some initial backups
under /etc/lvm and I would recommend backing these up to another location as well.
Once you've followed the initial setup instructions, i.e., you've followed the above Gentoo guide and parts of the Handbook for your arch, you might be wondering how to get back into a chroot with your lovely new RAID/LVM2 setup at some point, since you can't always just mount your raided volumes like you can an ordinary disk partition.
First, load the modules, for example, raid1 and dm-mod for LVM on a RAID 1 setup. Also, you should have used the magic "Linux raid autodetect" partition type when you created your partitions. Lastly, you'll need to assemble your RAID arrays and then activate your LVM volumes before you can mount them all and chroot. The following example commands will illustrate both. They are based on 4 raid pairs for /boot, /, swap, and everything else. The last "everything" partition contains several logical volumes, more or less following the Gentoo setup guide above.
Manual commands for starting a healthy RAID with LVM setup: (uses raid1 and lvm2 setup as an example)
livecd ~ # modprobe raid1
livecd ~ # modprobe dm_mod
(if you have the config file)
livecd ~ # mdadm --scan --config=/path/to/mdadm.conf
(if not, then this should work, as long as you specfiy the right devices)
livecd ~ # mdadm --assemble /dev/md1 /dev/hde1 /dev/hdg1
mdadm: /dev/md1 has been started with 2 drives.
livecd ~ # mdadm --assemble /dev/md2 /dev/hde2 /dev/hdg2
mdadm: /dev/md2 has been started with 2 drives.
livecd ~ # mdadm --assemble /dev/md3 /dev/hde3 /dev/hdg3
mdadm: /dev/md3 has been started with 2 drives.
livecd ~ # mdadm --assemble /dev/md4 /dev/hde4 /dev/hdg4
mdadm: /dev/md4 has been started with 2 drives.
livecd ~ # cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 hde1[0] hdg1[1]
80192 blocks [2/2] [UU]
md2 : active raid1 hde2[0] hdg2[1]
441664 blocks [2/2] [UU]
md4 : active raid1 hde4[0] hdg4[1]
114687936 blocks [2/2] [UU]
md3 : active raid1 hde3[0] hdg3[1]
2008000 blocks [2/2] [UU]
unused devices:
livecd ~ # lvs
No volume groups found
livecd ~ # vgchange -a y
No volume groups found
livecd ~ # vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
livecd ~ # vgchange -a y
6 logical volume(s) in volume group "vg" now active
You should now be able to mount your raid and lvm2 devices; for example, I would mount my root partition first (straight raid, not part of my logical volume) as /dev/md2, then mount the remaining devices under /dev/vg (where vg is whatever your volume group name is, as shown above). You can check this by looking at the symlinks in /dev/vg:
livecd ~ # ls -l /dev/vg/
total 0
lrwxrwxrwx 1 root root 19 May 12 08:46 home -> /dev/mapper/vg-home
lrwxrwxrwx 1 root root 18 May 12 08:46 opt -> /dev/mapper/vg-opt
lrwxrwxrwx 1 root root 18 May 12 08:46 tmp -> /dev/mapper/vg-tmp
lrwxrwxrwx 1 root root 18 May 12 08:46 usr -> /dev/mapper/vg-usr
lrwxrwxrwx 1 root root 18 May 12 08:46 var -> /dev/mapper/vg-var
In the event that you have a failed drive, follow the HOWTO process, remove and replace the failed drive, and boot your machine in degraded mode to sync your data to the new drive. Don't forget to re-create the same partition scheme on your new drive. Hint: you can use sfdisk to copy the partition table from one drive to another:
livecd ~ # sfdisk -d /dev/hda | sfdisk /dev/hdb
Just make sure the new drive is at least as big as the existing drive(s)...
Last updated 10/12/2008 19:54