09 November 2010

Configure RAID after Installation

RAID 0
You could consider RAID 0, also known asstriping, to be a bit misleading. It is actually not a redundant array at all. With a RAID 0 array you need at least two disks. Each write to this array is striped across both disks so that in effect the two drives become one large disk. So if you have two 100GB hard drives in a RAID 0 array, you will have 200GB of storage.While RAID 0 offers great speed, the downside is that there is no redundancy. If you lose a drive in a RAID 0 array, all of your data is lost.

RAID 1
Is also known as mirroring. In a RAID 1 array every bit that is written to one disk is copied to the other. As with RAID 0, RAID 1 requires at least two drives; however, in this case a RAID 1 array is only as big as one of its drives. So if you had two 100GB drives in a RAID 1 array, you would have 100GB of storage. The upside is that you could lose one of the drives in the array and still have all of your data.

RAID 5
RAID 5 is also known as striping plus parity. A RAID 5 array requires at least three drives. Every time the array is written to, the data is split across the three drives. In addition to the data, parity information is split among the drives so that any drive in the array can fail and not only will the remaining drives have all of the data, once the failed drive is replaced, the other drives can rebuild it. In a RAID 5 array you basically lose one drive’s worth of storage, so in a RAID 5 array of three 100GB disks you would have 200GB of storage. A RAID 5 array of four 100GB disks would have 300GB of storage.

NOTE
RAID 5 as a Root Partition It’s important to note that while GRUB can read a software RAID 1 array, it can’t read software RAID 5 arrays. This means that if you choose to have a RAID 5 array for your root partition, you must make a separate partition for the /boot directory that isn’t RAID 5 for GRUB to use. A common scenario for a three-disk RAID is a three-partition RAID 1 array for /boot and a three-partition RAID 5 array for the root directory.

So Let´s begin;

You may choose to add RAID storage to a server after the initial installation. For now I will assume you simply want to add a RAID array to an existing server. And for this example I will assume I have added three new drives, /dev/sdb, /dev/sdc, and /dev/sdd, and I want to partition the drives and create a RAID 5 array across all three partitions that I then mount at /mnt/storage.

Software RAID arrays are created and managed under Ubuntu with the mdadm  tool. This tool might not be installed by default; if it isn’t, run sudo apt-get install mdadm  to install the package. The next step is to partition each of these drives. In my case I just create a single partition that spans the full drive. Use whichever partitioning tool you prefer (like fdisk or cfdisk) as though you were creating any other partition. The only difference here is to change the partition type from the default of 82  to  fd. The fd  partition type is set aside for Linux RAID autodetect. If a partition is set to that type, it tells Linux that it is part of a software RAID array.

Once your partitions are set up, you can use mdadm on the command line to create the MD device. In the case of a RAID 1 array you would type:

$ sudo mdadm --create /dev/md0 --level=1 –raid-devices=2  /dev/sdb1 /dev/sdc1
mdadm: array /dev/md0 started.

Most of the arguments here are pretty self-explanatory. The --create option tells mdadm  that I want to create a new MD device at /dev/md0. If I already had a RAID array at /dev/md0, I would just pick the next number, such as /dev/md1. The --level; option sets which RAID level to use for this array, --raid-devices sets the number of active devices, and finally you specify each partition you want to use for the array.

In my case I want to set up a RAID 5 array across /dev/sdb1, /dev/sdc1, and /dev/sdd1, so I would type:

$ sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sdb1/dev/sdc1 /dev/sdd1
mdadm: array /dev/md0 started.

Once I have created the array, I can check out its current health in the /proc/mdstat file:

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd1[2] sdc1[1] sdb1[0]
16771584 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

unused devices: none

Now I can treat /dev/md0 like any other partition and format it with a file system of my choice and then mount it:

$ sudo mkfs -t ext3 /dev/md0
$ sudo mkdir /mnt/storage
$ sudo mount /dev/md0 /mnt/storage

Now this array is up and functioning; however, it is not yet set up to automatically start and mount at boot time. If you don’t set this up, you will have to run an mdadm; command to assemble the array along with a mount command each time the system boots. To start the array at boot time, you need to configure /etc/mdadm/mdadm.conf with details about your array. Now you could certainly do this by hand, but mdadm provides a simpler way. The mdadm --detail --scan; command will output an mdadm.conf compatible string for each of your arrays, so all you have to do is redirect that output to the /etc/mdadm/mdadm.conf file:

$ sudo sh -c 'mdadm --detail --scan >> /etc/mdadm/mdadm.conf'

Now edit your /etc/fstab and add an entry for /dev/md0 as though it were any other mount point. In my case I would add:

/dev/md0 /mnt/storage ext3 defaults 0 0

Alternatively, I could specify the UUID for this device in fstab as with the rest of the partitions. To figure that out I would type:

$ sudo blkid | grep /dev/md0
/dev/md0: UUID="99e190a7-dfe7-48ee-bf56-f426ef5343af" type="ext4"

Once /etc/mdadm/mdadm.conf and /etc/fstab are set up, I can reboot and then check /proc/mdstat to make sure the array comes up and then confirm it is mounted.


Software RAID Management

The bulk of the management of your software RAID arrays is done in two places: /proc/mdstat and mdadm. The /proc/mdstat file provides the current status of all of your running RAID arrays, including progress bars should any of them rebuild a disk. A standard /proc/mdstat file for a single RAID array might look like the following:

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd1[2] sdc1[1] sdb1[0]
16771584 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

unused devices: none

In the output you can see which array is active (md0), what RAID level it uses (raid5), and which partitions it is using (sdd1, sdc1, and sdb1). In the final line you can see that the RAID is healthy in that it has three out of three disks all active ([3/3] [UUU]). That section will change if any disks become faulty, as we see below.

While you can get status from /proc/mdstat, the bulk of the actual RAID management is done with mdadm. For instance, this tool can report basic and more complete information about an array with the --query and --detail; arguments respectively:

$ sudo mdadm --query /dev/md0
/dev/md0: 15.99GiB raid5 3 devices, 0 spares.

Use mdadm --detail for more detail.

$ sudo mdadm --detail /dev/md0

...
...

Replace a Failed Disk

While all of the information from mdadm can be useful, you will find you mostly use mdadm when a drive fails. When a drive fails, the mdadm daemon that runs on the system automatically sends an e-mail to the root user on the host. To change this, edit /etc/mdadm/mdadm.conf and locate the MAILADDR option in the file. After you save your changes, run sudo /etc/init.d/mdadm reload to load the new options. Apart from the e-mail you can also see that a drive has failed from /proc/mdstat:

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6]
[raid5] [raid4] [raid10]
md0 : active raid5 sdb1[0] sdd1[3](F) sdc1[1]
16771584 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

unused devices: none

Here you can see that sdd1  is marked with an (F) stating it has failed and on the third line of output the array shows two out of three disks ([3/2] [UU_]). The next step is to remove the disk from /dev/md0 so that I can swap it out with a new drive. To do this I run mdadm with the --remove option:

$ sudo mdadm /dev/md0 --remove /dev/sdd1

The drive must be set as a failed drive for you to remove it, so if for some reason mdadm hasn’t picked up the drive as faulty but you want to swap it out, you might need to set it as faulty before you remove it:

$ sudo mdadm /dev/md0 --fail /dev/sdd1

The mdadm command supports chaining commands, so you could fail and remove a drive in the same line:

$ sudo mdadm /dev/md0 --fail /dev/sdd1 --remove /dev/sdd1

Once you remove a drive from an array, it will be missing from /proc/mdstat:

$ cat /prod/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6]
[raid5] [raid4] [raid10]
md0 : active raid5 sdb1[0] sdc1[1]
16771584 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

unused devices: none

Now you can swap out the drive with a fresh one and partition it. Be sure that when you replace drives you create new partitions to be equal or greater in size than the rest of the partitions in the RAID array. Once the new partition is ready, use the --add command to add it to the array:

$ sudo mdadm /dev/md0 --add /dev/sdd1

Now mdadm will start the process of resyncing data. This can take some time, depending on the speed and size of your disks. You can monitor the progress from /proc/mdstat:

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6]
[raid5] [raid4] [raid10]
md0 : active raid5 sdd1[3] sdb1[0] sdc1[1]
16771584 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
[>. ..................]  recovery =  2.0% (170112/8385792)
finish=1.6min speed=85056K/sec

unused devices: none

Beyond this basic RAID management there are a number of different tweaks and customizations you can make to RAID arrays, particularly when you create them. For a full list of options and settings check out the mdadm manual (man mdadm).


See also:
Official Ubuntu Server Book, The (2nd Edition)




biOos

No comments: