USB stick RAID in ACTION!

Warning. Attempting similar projects at home may cause feelings of grandiosity.

Introduction

RAIDs (Redundant Arrays of Inexpensive [sic] Disks) are considered pretty handy for a number of things. This is an example of productive and practical use of a RAID. Granted, this project does not have the archaic grandeur of a Floppy Disk RAID, but then again, the capacity and performance of this system are utterly superior to those of a Floppy Disk RAID. The following is meant as an instruction sheet of how to build a rock-hard USB stick RAID system and simultaneously transform from an ordinary nerd to a SUPER LINUX GURU.

Ingredients

There is no unequivocal agreement between chefs de computing cuisine about the ingredients required for a USB stick RAID, but with this kind of a recipe one can easily make good use of a half an hour of time and about fifty hard-earned euros.

Necessary for the show:
  1. A computer with a USB port (no monitor or other sissy stuff needed)
  2. Gentoo Linux with some nerdtweaks
  3. A USB HUB
  4. Three (3) USB sticks: the sticks used in this utility are 128M sticks of different brands
  5. Single lifestyle

Set-up

Well, the hardware set-up is quick but you can still get your hands dirty: plug-in the HUB and the sticks and take a sip of Coke. However, because you're running GNU/Linux and not Windows, prepare to spend two days in figuring out how to make ALL the different brands of USB sticks work. Oh yeah, don't forget to compile the RAID modules to the kernel while doing your daily tweak.

Figure 1: Stick'em in

When you can make all the USB sticks flash their tiny led-eyes, you know you're ready to go.

First, tell the ordinary USB sticks to transform into mighty Linux Raid Autodetect sticks (notice the author's deep thought in naming his computer):

root@dragon mmeri # fdisk /dev/ubb

Command (m for help): p

Disk /dev/ubb: 131 MB, 131072000 bytes
32 heads, 32 sectors/track, 250 cylinders
Units = cylinders of 1024 * 512 = 524288 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/ubb1               1         250      127984   fd  Linux raid autodetect

Command (m for help): q
root@dragon mmeri #

Then, command your sticks to regroup into a RAID device. IMHO there's no difference between the coolness of different RAID levels: no matter which one you choose for your server, it tells equally good a story about the extent of your social life. So in the following we create (-C) verbosely (-v) a device (/dev/md0) with RAID level 5 (-l=5) out of three (-n=3) devices uba1, ubb1, and ubc1. Tell yourself, THIS ROCKS!

root@dragon mmeri # mknod /dev/md0 b 9 0
root@dragon mmeri # mdadm -C -v /dev/md0 -l=5 -n=3 /dev/uba1 /dev/ubb1 /dev/ubc1
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: size set to 125824K
mdadm: largest drive (/dev/ubb1) exceed size (125824K) by more than 1%
Continue creating array? Yes
mdadm: array /dev/md0 started.
root@dragon mmeri #

Now you've actually turned your three slack USB sticks into a SUPER COOL RAID device! A trick that leaves Copperfield dumbstruck and wins you the chicks. Trust me, I'm almost a doctor. Then, grab another 1.5L Coke (I know you've got one if you've read it this far), and format and mount your new beauty:

root@dragon mmeri # mke2fs /dev/md0
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
62992 inodes, 251648 blocks
12582 blocks (5.00%) reserved for the super user
First data block=1
31 block groups
8192 blocks per group, 8192 fragments per group
2032 inodes per group
Superblock backups stored on blocks:
        8193, 24577, 40961, 57345, 73729, 204801, 221185

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@dragon mmeri # mount /dev/md0 /mnt/usbraid/
root@dragon mmeri #

Drink the coke and relax reading hex dumps while the USB sticks on the HUB blink like a computer geek's star constellation (see the stunning movie1 or movie2). This'll take a while. When the formatting is finally done, observe the tremendous capacity that your new fault-tolerant device has (even more than this if you don't use RAID5) -- 231M should be enough for anybody! Also notice how the performance of your system KICKS ASS!

root@dragon mmeri # df
Filesystem           1K-blocks      Used Available Use% Mounted on
...
/dev/md0                243695        13    231100   1% /mnt/usbraid
...
root@dragon mmeri # hdparm -tT /dev/md0

/dev/md0:
 Timing cached reads:   744 MB in  2.00 seconds = 371.50 MB/sec
 Timing buffered disk reads:    4 MB in  4.08 seconds = 1004.07 kB/sec
root@dragon mmeri #

Sit back, take a sip of coke, and congratulate yourself. You are COOL DUDE.

Performance

This sorta record-cracking system surely cries to be tested for performance. The tools used in this test are special UNIX tools developed for system testing: date, cp, and umount. (Wizards time less, date more. And what's a hdparm?) First, we'll copy a file of size 221M on the RAID apparatus (write performance), while measuring the time, and then copy it back (read performance):

root@dragon mmeri # ls -l
total 2540661
...
-rwxr-xr-x  1 mmeri users 221832772 Dec 10  2004 VCDLuku1.mpg
...
root@dragon mmeri #date && cp VCDLuku1.mpg /mnt/usbraid/ && umount /mnt/usbraid && date
Sat Aug 13 14:35:20 EEST 2005
Sat Aug 13 14:50:43 EEST 2005
root@dragon mmeri # date && cp /mnt/usbraid/VCDLuku1.mpg /tmp/ && umount /mnt/usbraid && date
Sat Aug 13 15:27:57 EEST 2005
Sat Aug 13 15:31:51 EEST 2005
root@dragon mmeri #
As you can see, the write test raises the needle to fantastic 715k/sec, but the amazing thing comes with the second test. Our read performance test clocks a whopping 1M/sec, thus beating the competing technology, Floppy Disk RAID's 112k/sec hands down.

Conclusion

As RAID technology is meant only for serious computer-wizards who know stuff, the USB stick RAID will, unfortunately, remain a curiosity in the field of computing. Pity, especially if one considers the almost endless productive uses that this face-meltingly-fast technology could be harnessed for. Pack in the almost endless capacity, and you have a cool fault-tolerant data storage device.

Next in series: 5.25" Floppy disk RAID: With the inexpensive technology, is this a respectable competitor for USB stick RAID?

--By Daddy Kewl 13-Aug-2005

Appendix: Serious-looking stuff

root@dragon mmeri # mdadm -D /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Sat Aug 13 14:46:31 2005
     Raid Level : raid5
     Array Size : 251648 (245.75 MiB 257.69 MB)
    Device Size : 125824 (122.88 MiB 128.84 MB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sat Aug 13 16:50:43 2005
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 6cedba5f:f5643df2:2b3b4a07:b4d6b1c1
         Events : 0.95

    Number   Major   Minor   RaidDevice State
       0     180        1        0      active sync   /dev/uba1
       1     180        9        1      active sync   /dev/ubb1
       2     180       17        2      active sync   /dev/ubc1
root@dragon mmeri #