booting from a raid1

[lee]

Hi,

I'm trying to figure out how to boot from a RAID1 created with mdadm
on Debian testing amd64.

My starting point is a running system that has used an IDE disk for
all the system partitions (root, /usr, /var, /tmp ...) except
/home. Now I moved /home to a RAID5 made from a set of new disks, and
I want to use the disks that formerly stored /home for the system
partitions and thus replace the old IDE disk.

So far, I've partitioned two disks like this:


   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048   976773167   488385560   da  Non-FS data

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1            2048   976773167   488385560   da  Non-FS data


Then I created a RAID1 from these partitions:


md0 : active raid1 sdd1[0] sdf1[1]
      488384400 blocks super 1.2 [2/2] [UU]


The /dev/md0 device itself is partitioned as follows:


    Device Boot      Start         End      Blocks   Id  System
/dev/md0p1            2048   134219775    67108864   82  Linux swap / Solaris
/dev/md0p2   *   134219776   144705535     5242880   83  Linux
/dev/md0p3       144705536   354420735   104857600   83  Linux
/dev/md0p4       354420736   976768799   311174032    5  Extended
/dev/md0p5       354422784   564137983   104857600   83  Linux
/dev/md0p6       564140032   773855231   104857600   83  Linux
/dev/md0p7       773857280   976768799   101455760   83  Linux


Now /dev/md0p2 is to become the new root partition. To install grub,
I mounted /dev/md0p2 under /mnt/raid and ran:


grub-install --no-floppy --recheck --modules="raid mdraid" 
--root-directory=/mnt/raid/boot/ /dev/sdd
grub-install --no-floppy --recheck --modules="raid mdraid" 
--root-directory=/mnt/raid/boot/ /dev/sdf


This gave me a lot of warnings like the following for both physical
devices:


yun:~# grub-install --no-floppy --recheck --modules="raid mdraid" 
--root-directory=/mnt/raid/boot/ /dev/sdd
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos1).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos2).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos3).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd5,msdos1,msdos1,msdos1).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd5,msdos1,msdos1,msdos2).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd5,msdos1,msdos1,msdos3).
[...]
/usr/sbin/grub-probe: error: unknown filesystem.
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos1).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos2).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos3).
/usr/sbin/grub-probe: warn: Discarding improperly nested partition 
(hd5,msdos1,msdos1,msdos1).
[...]
/usr/sbin/grub-setup: warn: Discarding improperly nested partition 
(hd3,msdos1,msdos1,msdos3).
Installation finished. No error reported.
yun:~# 


Is grub unable to detect the partitions, and how comes that there's an
error about an unknown filesystem, but then it says "No error
reported."?

After running grub-install as above, I found that
/mnt/raid/boot/grub/grub.cfg contains entries that might be suitable
to use the partition on the IDE disk as a root partition rather than
the partition on the RAID1. So I tried to edit the file, but I was
unable to find documentation about the meaning of its contents. It
looks like this now:


#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
  load_env
fi
set default="0"
if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  insmod vbe
  insmod vga
  insmod video_bochs
  insmod video_cirrus
}

insmod part_msdos
insmod ext2
set root='(hd0,msdos3)'
search --no-floppy --fs-uuid --set 173bb2da-475b-4a4c-9948-c6455ff5b389
if loadfont /share/grub/unicode.pf2 ; then
  set gfxmode=640x480
  load_video
  insmod gfxterm
fi
terminal_output gfxterm
insmod part_msdos
insmod ext2
set root='(hd0,msdos1)'
search --no-floppy --fs-uuid --set 0e45de14-5c65-4892-877b-851d0761e6e6
set locale_dir=($root)/boot/grub/locale
set lang=en
insmod gettext
set timeout=5
### END /etc/grub.d/00_header ###


### BEGIN /etc/grub.d/10_linux ###
menuentry 'Debian GNU/Linux, with Linux 2.6.35.6-tst' --class debian --class 
gnu-linux --class gnu --class os {
        insmod part_msdos
        insmod ext2
        insmod raid
        insmod mdraid
#       set root='(hd0,msdos1)'
        search --no-floppy --fs-uuid --set 419b5497-294f-4a82-9ea8-5dc007f47e9d
        echo    'Loading Linux 2.6.35.6-tst ...'
        linux   /boot/vmlinuz-2.6.35.6-tst iommu=noaperture root=/dev/md0p2 ro  
}


What exactly do the searches (like search --no-floppy --fs-uuid --set
0e45de14-5c65-4892-877b-851d0761e6e6) do?

The UUIDs 173bb2da-475b-4a4c-9948-c6455ff5b389 and
0e45de14-5c65-4892-877b-851d0761e6e6 are or will become invalid
because the corresponding partitions or disks are or will be
removed. What do I do about that?

Can I just remove everything between "### BEGIN /etc/grub.d/00_header
###" and "### END /etc/grub.d/00_header ###"? (I don't care if there's
a background image or some particular font or not when grub displays
the boot menu, and 'safedefault' doesn't work work anyway ...).

Is a "set root='[something]'" required? If so, what do I need to set
it to? If I need to set it to either /dev/sdd or /dev/sdf, how do I
know which (hdX) grub considers as these? What if the disk specified
fails?

What exactly does "root" mean to grub? I can see it for physical
devices, but in this case, there's /dev/md0 in place of a physical
device. How about "root=(/dev/md0,msdos2)"?

Are there other partition types (or whatever it is) than "msdos"?
Fdisk tells me to turn off msdos compatibility when partitioning
disks, so I suspect that grub might be unable to understand the
partitioning information ...

How do I prevent the system from becoming unbootable in case I change
the cabling and plug /dev/sdd or /dev/sdf or both into different ports
of the SATA controllers? The RAID arrays easily survive that because
they go by UUIDs, but how about grub?

How do I make it so that grub will boot the kernel from /dev/md0p2?

All the partitions from the IDE disk have been copied over to the
RAID1 already. The only still in use on that disk is the root
partition. Once I can solve the booting problem, I can retire that old
IDE disk. Any help would be appreciated.