(This is part 1 of a couple part article. I'm only sectioning it off because I can get very long winded at times.)
Here's what I used to put this together:
* PIII-866 512MB RAM (Got mine for free at the local school)
* 40gb IDE boot harddrive
* Any Linux OS (Free, I used Fedora Core 7)
* 2 - Promise ATA-100 PCI EIDE Controllers
* 3 - Western Digital WD1200 120GB IDE harddrives
* Intel Express E100 10/100 PCI NIC
I had all this junk lying around my house, so it really cost me nothing. I, myself, do not need some 2TB NAS on my network, so again, this is the thrifty approach here. Most of this stuff I've gotten for free over the years anyway, so if you're constantly in the used computer hardware market like me, you'll have no problems putting something like this together and not spending much of anything do it. It's pretty obvious I don't throw much stuff away either...
Server Assembly
This is more of a cautionary-slash-design note than anything. You might be asking why I picked to use 2 ATA controllers instead of just one (e.g. I have 3 drives, and with a primary and secondary controller on one card, that allows 4 total devices)? The reason is performance and reliability. If you have two IDE disks on the same controller chain, you'll see a performance hit. The reliability factor is that if you happen to have a bad disk or bad controller, you're going to loose two drives instead of one because of a failure. Since I'm planning on using a RAID-5 configuration without a hot spare, putting the individual drives each on an individual controller made the most sense to me. However, I also plan to contrast the differences in using RAID-0, which it's only advantage for me would be speed verses redundancy to the point with RAID-5 that I'd have time to go get another drive and replace it and n-1 concept of the parity drive and eating 120gb in the pants.
Harddrive Discovery and Optimization
After the box was assembled and the Linux OS is installed, first thing to do is to gather the device names of all the disks you are going to use in your RAID configuration. I'm looking for my 3 Western Digital drives that live on my Promise ATA controllers. Best way to do this is simply do some fancy grepping on the `dmesg` output and then look for your disks:
[testbox]& dmesg | grep ^hd[a-z]:
hde: WDC WD1200BB-00CAA2, ATA DISK drive
hdg: WDC WD1200BB-00CAA1, ATA DISK drive
hdi: WDC WD1200BB-00DWA0, ATA DISK drive
hda: Maxtor 32305U3, ATA DISK drive
hdc: IDE DVD-ROM 16X, ATAPI CD/DVD-ROM drive
hde: max request size: 128KiB
hde: 234493056 sectors (120060 MB) w/2048KiB Cache, CHS=16383/255/63, UDMA(100)
hde: cache flushes supported
hdg: max request size: 128KiB
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdg: cache flushes not supported
hdi: max request size: 128KiB
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=16383/255/63, UDMA(100)
hdi: cache flushes supported
hda: max request size: 128KiB
hda: 45023328 sectors (23051 MB) w/512KiB Cache, CHS=44666/16/63, UDMA(100)
hda: cache flushes not supported
hdc: ATAPI 44X DVD-ROM drive, 512kB Cache, UDMA(33)
The disks I'm looking for (which are in bold) are the Western Digital WD1200's I want use for my RAID-5 configuration.
Next, you'll want to check to see what type of disk performance you're going to be looking at getting. To do this, you'll definitely want to make sure you have the `hdparm` package installed. In Fedora, to install this package if you don't have it, do the following:
[testbox]& yum install hdparm -y
`hdparm` is a pretty powerful hard disk ioctl tool, that gives you direct access into tweaking disk features, along with a very rough, but somewhat accurate read/buffer testing.
Right away when I first booted up the box with the Western Digital drives, I ran a buffered read and cache test on the drives, and noticed on a few of them that I was getting some really poor-ass performance:
[testbox]& hdparm -Tt /dev/hde
/dev/hde:
Timing cached reads: 992 MB in 2.00 seconds = 494.97 MB/sec
Timing buffered disk reads: 12 MB in 3.00 seconds = 4.00 MB/sec
...which is terrible theoretically considering ATA-5 drives on an ATA-100 controller should get, at best, roughly 50-60mb/sec buffered disk reads from my experience. So how'd I solve this? With one acronym: DMA (e.g. Direct Memory Access). DMA is essentially allowing your EIDE devices (such as a harddrive or CDROM) to bypass your microcontroller and have direct transfer to your computer's memory.
So, let's check out what this particular drive has set on it by using `hdparm`:
[testbox]& hdparm /dev/hde
/dev/hde:
multcount = 16 (on)
IO_support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 0 (off)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 16383/255/63, sectors = 234493056, start = 0
...notice that DMA is turned off (in bold above). To enable DMA, simply type:
[testbox]& hdparm -d1 /dev/hde
/dev/hde:
setting using_dma to 1 (on)
using_dma = 1 (on)
Now that DMA is toggled on, let's see what we get for buffered disk and cache reads:
[testbox]& hdparm -Tt /dev/hde
/dev/hde:
Timing cached reads: 956 MB in 2.00 seconds = 476.95 MB/sec
Timing buffered disk reads: 170 MB in 3.03 seconds = 56.12 MB/sec
Not bad! ~56mb/sec is a much improvement over ~4mb/sec. Again, check the rest of your drives that you are using for your RAID setup and make sure DMA is enabled. Obviously DMA is just one of many options, but it's definitely the first and foremost thing you want to address right away.
We'll end there, even though we didn't get a whole lot covered. Next, I'll dive into setting up a software RAID-5 configuration using the `mdadm` tools, compare it with a RAID-0 configuration and decide which might be best for my needs, write a nice EXT3 filesystem onto it and benchmark away!
