HOWTO: Installing on Intel (imsm) fakeraid using mdadm
I finally discovered the reason for my fakeraid's sudden breakage. It seems that mdadm now can deal with Intel's fakeraid (ICHxR.) Apparently when both dmraid and mdadm are used, really strange things happen. It seems that dmraid won't assemble the array if mdadm picks it up first. dmraid is smart enough to say that the raid is already assembled and in use, but (of course) it doesn't populate /dev/mapper nor tell you it's located in /dev/md/* and you need to use mdadm.
Lots of headaches for me on this one. I did manage to get a system up and running, dual-booting and all, in about three weeks of on-and-off experimenting. Thankfully I had a laptop or I would've been more upset.
Intel is now fully supporting mdadm and not recommending the use of dmraid. mdadm-3.2.1 contains a lot of updates for imsm raids, you may have to unmask it to build that version.
Known Issues / Notes
1. NOTE: dm-raid and mdraid can conflict with each other! Use one or the other! This guide, of course, is for using mdadm.
2. This should be obvious, but your install may not use the same device files as mine. Substitute them accordingly.
Prerequisites
These are the items you will need that you'll need to prepare:
I finally discovered the reason for my fakeraid's sudden breakage. It seems that mdadm now can deal with Intel's fakeraid (ICHxR.) Apparently when both dmraid and mdadm are used, really strange things happen. It seems that dmraid won't assemble the array if mdadm picks it up first. dmraid is smart enough to say that the raid is already assembled and in use, but (of course) it doesn't populate /dev/mapper nor tell you it's located in /dev/md/* and you need to use mdadm.
Lots of headaches for me on this one. I did manage to get a system up and running, dual-booting and all, in about three weeks of on-and-off experimenting. Thankfully I had a laptop or I would've been more upset.
Intel is now fully supporting mdadm and not recommending the use of dmraid. mdadm-3.2.1 contains a lot of updates for imsm raids, you may have to unmask it to build that version.
Known Issues / Notes
1. NOTE: dm-raid and mdraid can conflict with each other! Use one or the other! This guide, of course, is for using mdadm.
2. This should be obvious, but your install may not use the same device files as mine. Substitute them accordingly.
Prerequisites
These are the items you will need that you'll need to prepare:
Latest gentoo minimal livecd (yes, it uses mdadm and appears to find the array)
Ubuntu 11.x installer disc (may be needed for grub install)
Go into the IMSM BIOS and configure an array. (While you can use mdadm to do this, it's far easier this way.)
Getting the system installed
Boot the latest gentoo minimal CD.
Boot the default kernel with domdadm in the kernel boot line.
| Code: |
> gentoo domdadm nodmraid |
During the boot process mdraid will do a scan on devices.
To determine features, use `mdadm --detail-platform`:
| Code: |
livecd ~ # ls /dev/md HDD_0 imsm0 # This is a good sign! Both the Intel fakeraid container and the array created were found. livecd ~ # mdadm --detail-platform Platform : Intel(R) Matrix Storage Manager Version : 8.9.1.1002 RAID Levels : raid0 raid1 raid10 raid5 # The controller supports these raid levels through mdadm. Chunk Sizes : 4k 8k 16k 32k 64k 128k Max Disks : 6 Max Volumes : 2 I/O Controller : /sys/devices/pci0000:00/0000:00:1f.2 Port0 : /dev/sda (5QM18SWD) Port1 : /dev/sdb (5QM1AAG3) Port2 : /dev/sdc (5QM1BX8S) Port3 : /dev/sdd (5QM1BFL5) Port4 : - non-disk device (TSSTcorp CDDVDW SH-S203B) - Port5 : - no device attached - |
mdadm should locate the array and create /dev/md/* entries. If it doesn't do this automatically:
| Code: |
livecd ~ # mdadm --assemble --scan livecd ~ # mdadm -I /dev/md/imsm0 ARRAY metadata=imsm UUID=8f6a20d5:66434ba8:2c1c922f:3985dbca ARRAY /dev/md/HDD container=8f6a20d5:66434ba8:2c1c922f:3985dbca member=0 UUID=d2a4df36:7388143b:88d727ca:51491287 |
With that, the arrays are assembled. Time to see what device files to use:
| Code: |
livecd ~ # ls -l /dev/md/ total 0 lrwxrwxrwx 1 root root 8 Jul 25 16:10 HDD_0 -> ../md126 lrwxrwxrwx 1 root root 8 Jul 25 16:11 imsm0 -> ../md127 |
The imsm0 is the "container" for the actual raid configuration. You can have several raids inside one container (as an example, a raid10 for / and raid5 for /home, but they need to span the same disks.) The HDD_0 above is a RAID10 array that was created (and labeled 'HDD') in the Intel BIOS.
At this point the array(s) should build themselves. The Intel BIOS only creates the container and arrays but does not actually assemble them:
| Code: |
livecd ~ # cat /proc/mdstat Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md126 : active raid10 sda[3] sdb[2] sdc[1] sdd[0] 976768000 blocks super external:/md127/0 64K chunks 2 near-copies [4/4] [UUUU] [============>........] resync = 63.9% (624938496/976768256) finish=72.9min speed=80326K/sec md127 : inactive sdd[3](S) sda[2](S) sdb[1](S) sdc[0](S) 9028 blocks super external:imsm unused devices: <none> |
You can work on the system while it's being built.
To partition, use:
| Code: |
| fdisk /dev/md/HDD_0 |
OR
| Code: |
| fdisk /dev/md126 |
Partitions will appear as /dev/md126p? in this example:
| Code: |
livecd ~ # ls -l /dev/md* brw-rw---- 1 root disk 9, 126 Jul 25 16:10 /dev/md126 brw-rw---- 1 root disk 259, 0 Jul 25 16:08 /dev/md126p1 brw-rw---- 1 root disk 259, 1 Jul 25 17:15 /dev/md126p2 brw-rw---- 1 root disk 259, 2 Jul 25 17:14 /dev/md126p3 brw-rw---- 1 root disk 259, 3 Jul 25 17:16 /dev/md126p4 brw-rw---- 1 root disk 9, 127 Jul 25 16:11 /dev/md127 |
Create filesystems on these and start the gentoo manual install as you normally would, but with these tips:
(All these tips are for when you are chrooted in to your new gentoo install.)
1. Ensure you use the new /dev/md126p? entries in /etc/fstab. fdisk might report something else, but they don't work (or didn't when I tried.)
2. Configure and install mdadm! It's better to install mdadm before configuring the kernel.
| Code: |
livecd ~ # emerge mdadm livecd ~ # echo 'DEVICE /dev/sda /dev/sdb /dev/sdc /dev/sdd' >> /etc/mdadm.conf livecd ~ # mdadm --detail --scan >> /etc/mdadm.conf |
Replace the devices for the ones used in your array.
3. Configure the kernel. Install mdadm first, see step 2.
Use genkernel, it's the easiest way; you can edit /etc/genkernel.conf and tell it to use mdadm (required) and pop up menuconfig so you can configure your kernel if you like (optional.) Also, you can use genkernel to build an initramfs that has a newer version of mdadm if you need. This is recommended. Improvements for imsm containers go in every release.
See below:
| Code: |
# Run 'make menuconfig' before compiling this kernel? MENUCONFIG="yes" # Includes mdadm/mdmon binaries in initramfs. # Without sys-fs/mdadm[static] installed, this will build a static mdadm. MDADM="yes" |
Then:
| Code: |
livecd ~ # genkernel all |
...and then go get a coffee.
When it's done, add the new kernel entries to /boot/grub/grub.conf. Very important: Make sure you add 'domdadm' (without quotes) to your kernel line!
4. Set up gentoo to shut down mdadm properly. (Bug #395203, post there or vote it up...)
BIG FAT WARNING: This step is very important! If you skip this step, mdadm will not shut down properly, causing the IMSM metadata to be put in a dirty state. This will cause your array to fully rebuild on each reboot!
I discovered some issues with the way gentoo shuts down mdadm. The IMSM metadata must be marked properly before shutdown/reboot, and by default, gentoo terminates all process (including mdmon, which is responsible for marking the array as clean at the end of the shutdown process.) killprocs terminates the mdmon process and continues its shutdown process (including doing writing) after mdmon is forced to terminate. This causes the array to be in a degraded state! While I had an almost-working solution, with some help below there's a fully working solution for this.
Short explanation: We need to prevent killprocs from terminating mdmon until AFTER root is marked as read-only.
To do this, install openrc-0.9.4 or higher (the following command will install 0.9.4 specifically):
| Code: |
# emerge =sys-apps/openrc-0.9.4 |
After it's installed, edit /etc/conf.d/killprocs (this command takes into accounts multiple process IDs and delineates them accordingly):
| Code: |
# If you wish to pass any options to killall5 during shutdown, # you should do so here. killall5_opts="`pidof mdmon | sed -e 's/ /,/g' -e 's/./-o \0/'`" |
The backticks are important, don't remove them!
After that, create a new initscript /etc/init.d/mdadm-shutdown:
| Code: |
!/sbin/runscript depend() { after mount-ro } start() { ebegin 'Shutting down mdadm' mdadm --wait-clean --scan --quiet eend $? } |
Save the changes, and set the new initscript as executable and add it to the shutdown runlevel:
| Code: |
# chmod +x /etc/init.d/mdadm-shutdown # rc-update add mdadm-shutdown shutdown |
This step is now done.
5. Go through the install guide and set everything up, including configuring /boot/grub/grub.conf. Skip the installing grub part for now, see below.
When you are completely done configuring everything, exit the chroot and boot the Ubuntu livecd.
Installing grub
OK, let's face it, grub is a pain in the ass most times with raid devices (or anything even *slightly* unusual.)
Try this first
Grub looks for [devicename][partition number] when installing itself, so since the partitions are named mdXXXpY, Grub won't find them and give you error 22.
Simply create a symlink excluding the p to your boot partition and everything will work just fine:
| Code: |
| ~ # ln -s /dev/md126p2 /dev/md1262 |
Then install grub:
| Code: |
~ # grub --no-floppy grub> device (hd0,1) /dev/md1262 grub> device (hd0) /dev/md126 grub> root (hd0,1) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0) Checking if "/boot/grub/stage1" exists... yes Checking if "/boot/grub/stage2" exists... yes Checking if "/boot/grub/e2fs_stage1_5" exists... yes Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 18 sectors are embedded. succeeded Running "install /boot/grub/stage1 (hd0) (hd0)1+18 p (hd0,1)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded Done. |
After that the install is done. Leave the chroot and reboot.
If the above does not work: The ubuntu method for installing grub
I initially could not get grub to install using the gentoo livecd. I did find a workaround: boot into an Ubuntu livecd, which will use dmraid to find your array.
| Code: |
ubuntu ~ # sudo su - ubuntu ~ # dmraid -ay ubuntu ~ # ls -l /dev/mapper ubuntu ~ # fdisk -l /dev/mapper/isw_deaihjaddd_HDD Disk /dev/mapper/isw_deaihjaddd_HDD: 1000.2 GB, 1000210432000 bytes 255 heads, 63 sectors/track, 121602 cylinders, total 1953536000 sectors --snip-- |
(Make a note of the cylinders, heads, sectors - you'll need them in grub in a minute.)
| Code: |
ubuntu ~ # grub --no-floppy grub> device (hd0,1) /dev/mapper/isw_deaihjaddd_HDD2 grub> device (hd0) /dev/mapper/isw_deaihjaddd_HDD grub> geometry (hd0) 121602 255 63 #From the fdisk output above grub> root (hd0,1) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0) Checking if "/boot/grub/stage1" exists... yes Checking if "/boot/grub/stage2" exists... yes Checking if "/boot/grub/e2fs_stage1_5" exists... yes Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 18 sectors are embedded. succeeded Running "install /boot/grub/stage1 (hd0) (hd0)1+18 p (hd0,1)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded Done. |
Exit grub, the chroot, and reboot. You should see grub boot in all its glory.
Of course, your device files may not match the ones used in this guide, substitute them accordingly.
ليست هناك تعليقات:
إرسال تعليق