Date: Sun, 3 Jun 2001 19:46:25 -0700 (PDT)
From: Jason
To: Ryan
Subject: Re: warez raid


Sorry, this email is pretty long.  It does a walkthrough of everything you
have to do to get your raid up.


> * promise ide card with two controllers
> * 4 80 gb drives on the promise card
> * 2 30 gb drives on the on board controller
>
> i want to raid 5 the 80 gb drives together
> and raid 1 the 30 gb drives
> i don't care if i have to leave unraided
> partitions on each disk

Unless you do some magic that I haven't quite worked out yet, you have to
have an unraided partition to boot off of.  The magic isn't so hard, I
just haven't worked it out yet.  I usually bite 1G off of all the drives
and install the system onto just those, then once the system is installed,
I raid the rest of it and move usr and var down into the raid.  (Check out
walter for referrence.)


> * does each drive need it's own ide controller or
>   am i ok with two drives per controller?

You _can_ do two drives per controller, it will work, but especially with
raid, the performance will be pretty crappy.  Unlike SCSI, IDE performance
totally degrades when the bus is shared, and since it's always goind to be
writing to both the disks on the controller at the same time, it will be
pretty bad.

Since the Promise Ultra66 controllers are only $30 at Fry's, I'd just buy
another if there's room in the case.


> * any special kernel configuration required?

No, vinum is always built by default - just add start_vinum="YES" to your
/etc/rc.conf so that the module will get loaded at boot time.


> * how do i set up the raid once i have all the disks
>   set up and recognized?

Check out vinum(8), vinum(4) and http://ww.vinumvm.org for details.
In a nutshell:

First, use disklabel to change the partition types to vinum.  Eg, the
first disk is /dev/ad4s1 - run "disklabe -e /dev/ad4s1" - you'll be thrown
into vi with a description of the disklabel - at the bottom, it will look
like:

#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  2097152        0    4.2BSD     1024  8192    16   # (Cyl.    0 - 130*)
  c: 90044262        0    unused        0     0         # (Cyl.    0 - 5604*)
  e: 87947110  2097152    4.2BSD                        # (Cyl.  130*- 5604*)

Change the partition you're going to raid to fstype "vinum":

#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  2097152        0    4.2BSD     1024  8192    16   # (Cyl.    0 - 130*)
  c: 90044262        0    unused        0     0         # (Cyl.    0 - 5604*)
  e: 87947110  2097152     vinum                        # (Cyl.  130*- 5604*)

Repeat for all disks.


Next, create /etc/vinum.conf looking something like this:

 -------------------------------------
drive r1 device /dev/ad4s1e
drive r2 device /dev/ad5s1e
drive r3 device /dev/ad6s1e
drive r4 device /dev/ad7s1e
drive m1 device /dev/ad0s1e
drive m2 device /dev/ad2s1e
volume usr
	plex org raid5 256k
		sd length 0 drive r1
		sd length 0 drive r2
		sd length 0 drive r3
		sd length 0 drive r4
volume var
	plex org concat
		sd length 0 drive m1
	plex org concat
		sd length 0 drive m2
 -------------------------------------

"drive" directives just create symbolic names for the physical devices,
and they can be whatever you want.  Adjust to fit your actual device
names.  A "volume" is a virtual disk - the thing you're actually going to
mount when this is all done.  Inside a volume are "plex"'es - a plex is a
raid set - it's made up of drives that are raided together.  Multiple
plexes in a volume are mirrored together and therefore have to be the same
size.  So in this config, we're creating a volume called usr that has four
disks in a raid five (with 256k stripes), and we're using all available
space on each of them (the "length 0").  Then we have another volume
called var with two disks mirrored (again, using all available space on
those disks).  The names of the drives and the volumes can be whatever you
want - I'm just assuming here that you'll mount the raid5 as /usr for all
you homes, packages, and warez, and that the mirror will be /var, to
protect all your logs and your qmail queue.

Then you run something like "vinum create -f /etc/vinum.conf" to create
the arrays, then "vinum init" to initialize everything.  "vinum list" will
show you the status of everything, including the percent done on
initializations.  If a particular part fails to go into state "up",
initialize just that part.  For example, to just initialze the last disk
of the raid5, "vinum init usr.p0.s3", or to just initialize the second
disk in the mirror "vinum init var.p1.s0"

Finally, when "vinum list" says that everything is in state "up", create
filesystems, and go.

newfs -v /dev/vinum/usr		 # create filesystem - the "-v" is necesary
newfs -v /dev/vinum/var		 #    when the device isn't a physical disk
tunefs -n enable /dev/vinum/usr  # turn on soft updates for speed and to
tunefs -n enable /dev/vinum/var  #    protect in the event of a crash
mount /dev/vinum/usr /usr-raid   # and finally, mount them someplace, rsync
mount /dev/vinum/var /var-mirror #    them, move the old ones out of the
				 #    way, move the new ones into place and
				 #    reboot


> and one more thing:
> are there reiserfs drivers for bzd (so i can get
> data off of some linux drives)?

Sadly, no.  This is why I switched to ext3 on my linux boxes....  On the
other hand, linux can read and write bsd partitions (if you compiled with
CONFIG_UFS_FS, CONFIG_UFS_FS_WRITE, and CONFIG_BSD_DISKLABEL) so you can
push stuff from linux to bsd.  Oh, except linux doesn't know about
vinum....  Oh well.


 -Jason

 ---------------------------
 If the Revolution comes to grief, it will be because you and those you
 lead have become alarmed at your own brutality.         --John Gardner