17.4 RAID

17.4.1 軟體 RAID

17.4.1.1 連接式磁碟裝置驅動程式(CCD, Concatenated Disk Driver) 設定

Original work by Christopher Shumway. Revised by Jim Brown.

對大容量儲存設備而言,最關鍵的要素乃是速度、可靠性及價格。 然而這三者往往難以兼顧:快速可靠的設備通常很貴; 而降低成本通常也犧牲了速度或可靠性。

接下來要介紹的系統,價格是最重要的考量,接下來是速度,最後才是可靠性。 順序如此是因為資料傳輸的速度最終取決於網路,而儘管可靠性十分重要, 卻有簡單的取代方案:將資料完整備份於 CD-R 中。

選擇大容量儲存設備方案時,首先要定義您的需求。如果您重視速度或可靠性 甚於價格,接下來的介紹恐非您所需。

17.4.1.1.1 安裝硬體

除了系統磁碟外,下面介紹的 CCD 磁碟陣列將使用到三顆 30GB、 5400 RPM 的 Western Digital IDE 磁碟,以提供約 90GB 的儲存空間。 最理想的情況是每個磁碟由獨立使用的排線連接獨立使用的 IDE 控制器, 不過為了降低成本,利用 jumper 設定磁碟,使每個 IDE 控制器可連接 一個主磁碟加一個副磁碟,如此可不必加裝額外的 IDE 控制器。

開機後,BIOS 應該設定成自重偵測磁碟。更重要的是 FreeBSD 應該 要偵測到它們:

ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33
ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave UDMA33
ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33
ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave UDMA33

Note: 如果 FreeBSD 沒有偵測到所有磁碟,請確認 jumper 都設定正確。 許多 IDE 磁碟可以設定成 “Cable Select”(根據排線位置決定), 這並非 master(主磁碟) 或 slave(副磁碟)。請參閱磁 碟的說明文件以正確設定 jumper。

接下來,考慮如何將它們變成檔案系統的一部份。您可以參考 vinum(8)(Chapter 19) 及 ccd(4)。 在此我們選擇 ccd(4)

17.4.1.1.2 Setting Up the CCD

The ccd(4) driver allows you to take several identical disks and concatenate them into one logical file system. In order to use ccd(4), you need a kernel with ccd(4) support built in. Add this line to your kernel configuration file, rebuild, and reinstall the kernel:

device   ccd

The ccd(4) support can also be loaded as a kernel loadable module.

To set up ccd(4), you must first use bsdlabel(8) to label the disks:

bsdlabel -r -w ad1 auto
bsdlabel -r -w ad2 auto
bsdlabel -r -w ad3 auto

This creates a bsdlabel for ad1c, ad2c and ad3c that spans the entire disk.

The next step is to change the disk label type. You can use bsdlabel(8) to edit the disks:

bsdlabel -e ad1
bsdlabel -e ad2
bsdlabel -e ad3

This opens up the current disk label on each disk with the editor specified by the EDITOR environment variable, typically vi(1).

An unmodified disk label will look something like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)

Add a new e partition for ccd(4) to use. This can usually be copied from the c partition, but the fstype must be 4.2BSD. The disk label should now look something like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)
  e: 60074784        0    4.2BSD        0     0     0   # (Cyl.    0 - 59597)

17.4.1.1.3 Building the File System

Now that you have all the disks labeled, you must build the ccd(4). To do that, use ccdconfig(8), with options similar to the following:

ccdconfig ccd0(1) 32(2) 0(3) /dev/ad1e(4) /dev/ad2e /dev/ad3e

The use and meaning of each option is shown below:

(1)
The first argument is the device to configure, in this case, /dev/ccd0c. The /dev/ portion is optional.
(2)
The interleave for the file system. The interleave defines the size of a stripe in disk blocks, each normally 512 bytes. So, an interleave of 32 would be 16,384 bytes.
(3)
Flags for ccdconfig(8). If you want to enable drive mirroring, you can specify a flag here. This configuration does not provide mirroring for ccd(4), so it is set at 0 (zero).
(4)
The final arguments to ccdconfig(8) are the devices to place into the array. Use the complete pathname for each device.

After running ccdconfig(8) the ccd(4) is configured. A file system can be installed. Refer to newfs(8) for options, or simply run:

newfs /dev/ccd0c

17.4.1.1.4 Making it All Automatic

Generally, you will want to mount the ccd(4) upon each reboot. To do this, you must configure it first. Write out your current configuration to /etc/ccd.conf using the following command:

ccdconfig -g > /etc/ccd.conf

During reboot, the script /etc/rc runs ccdconfig -C if /etc/ccd.conf exists. This automatically configures the ccd(4) so it can be mounted.

Note: If you are booting into single user mode, before you can mount(8) the ccd(4), you need to issue the following command to configure the array:

ccdconfig -C

To automatically mount the ccd(4), place an entry for the ccd(4) in /etc/fstab so it will be mounted at boot time:

/dev/ccd0c              /media       ufs     rw      2       2

17.4.1.2 The Vinum Volume Manager

The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. vinum(8) implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination.

See Chapter 19 for more information about vinum(8).

17.4.2 Hardware RAID

FreeBSD also supports a variety of hardware RAID controllers. These devices control a RAID subsystem without the need for FreeBSD specific software to manage the array.

Using an on-card BIOS, the card controls most of the disk operations itself. The following is a brief setup description using a Promise IDE RAID controller. When this card is installed and the system is started up, it displays a prompt requesting information. Follow the instructions to enter the card's setup screen. From here, you have the ability to combine all the attached drives. After doing so, the disk(s) will look like a single drive to FreeBSD. Other RAID levels can be set up accordingly.

17.4.3 Rebuilding ATA RAID1 Arrays

FreeBSD allows you to hot-replace a failed disk in an array. This requires that you catch it before you reboot.

You will probably see something like the following in /var/log/messages or in the dmesg(8) output:

ad6 on monster1 suffered a hard error.
ad6: READ command timeout tag=0 serv=0 - resetting
ad6: trying fallback to PIO mode
ata3: resetting devices .. done
ad6: hard error reading fsbn 1116119 of 0-7 (ad6 bn 1116119; cn 1107 tn 4 sn 11)\\
status=59 error=40
ar0: WARNING - mirror lost

Using atacontrol(8), check for further information:

# atacontrol list
ATA channel 0:
    Master:      no device present
    Slave:   acd0 <HL-DT-ST CD-ROM GCR-8520B/1.00> ATA/ATAPI rev 0

ATA channel 1:
    Master:      no device present
    Slave:       no device present

ATA channel 2:
    Master:  ad4 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
    Slave:       no device present

ATA channel 3:
    Master:  ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
    Slave:       no device present

# atacontrol status ar0
ar0: ATA RAID1 subdisks: ad4 ad6 status: DEGRADED
  1. You will first need to detach the ata channel with the failed disk so you can safely remove it:

    # atacontrol detach ata3
    
  2. Replace the disk.

  3. Reattach the ata channel:

    # atacontrol attach ata3
    Master:  ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5
    Slave:   no device present
    
  4. Add the new disk to the array as a spare:

    # atacontrol addspare ar0 ad6
    
  5. Rebuild the array:

    # atacontrol rebuild ar0
    
  6. It is possible to check on the progress by issuing the following command:

    # dmesg | tail -10
    [output removed]
    ad6: removed from configuration
    ad6: deleted from ar0 disk1
    ad6: inserted into ar0 disk1 as spare
    
    # atacontrol status ar0
    ar0: ATA RAID1 subdisks: ad4 ad6 status: REBUILDING 0% completed
    
  7. Wait until this operation completes.

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.