How Many of What Disks?
| The third and fourth major hardware questions are:
These two questions should be resolved together. Gone for good are the days of just buying SCSI controllers and disks for servers. Today's data warehouses are far too big for that kind of solution ”they must be built using disk arrays. A disk array is a cabinet that houses front-end and back-end intelligent controllers, a sizable memory cache, and a very large disk farm (Figure 3-7). There are two kinds of disk arrays:
Figure 3-7. Basic Disk Farm Hardware Architecture A SAN is a disk farm appliance. It's a shared, dedicated, high-speed network connecting servers and disks, typically via fiber channel. I've heard SAN referred to as "simply a bunch of disks strung together with a bit of fiber optics." The underlying technology is SCSI. A NAS is a disk farm on a network. It's a special-purpose server with its own embedded software for file sharing across the network. I've heard NAS referred to as "just a RAID array with an Ethernet card." The underlying technology is network file system (NFS). There are great debates between both camps on whether SAN or NAS is better. I don't subscribe to these arguments for I've used both technologies without incident on large data warehouses. I can say that given a choice, I've found SAN to offer better performance with the large block sizes, high throughput, and write- intensive nature of large data warehouses. I also prefer NAS's centralized manageability and configuration flexibility. But, these differences are blurring as the lines between these two technologies continue to fade. Probably the best-known SAN disk arrays are from EMC Corporation. But vendors such as Sun, HP, Compaq, and IBM have offerings as well. Be forewarned that disk arrays are quite expensive. It's not uncommon for a disk array to cost as much or more than a database server. Often this high cost is in direct proportion to the cache size as disk arrays use very expensive high-speed memory. But, since the disk drives in them are often nothing more than next -generation PC server SCSI drives , the cache memory size is critical for obtaining acceptable performance. It's recommended that for big data warehouses, the disk array's cache be anywhere from 2 “8 GB. When selecting a disk array, make sure that it is scalable along four criteria. First, it must provide enough front-end controllers for the number of servers you may need to connect. Second, it must offer an initial and maximum cache memory size that will handle your maximum expected I/O needs while maintaining a 40% or better cache hit ratio. Third, it must provide sufficient bays for the number and size of disk drives that you desire to use. And finally, the disk array should be able to interconnect with other disk arrays from the same manufacturer or other vendors. It may sound funny , but with today's relatively cheap disk space costs, data warehouses are growing to multi-terabyte sizes and beyond. Often, it will take more than one disk array cabinet to hold that much data. If all that were not enough, most disk arrays support RAID, so you need to choose a RAID level to employ . The typical choices are RAID 0, 1, 0+1, and 3 “5, where:
In summary, the various RAID levels with their pros and cons are shown in Table 3-2. Table 3-2. Comparison of RAID Levels
Another way to view this is that for high write rates (i.e., redo logs, rollback segments, and temporary segments), RAID 0+1 is the best possible HA solution. RAID 5's writes are just too expensive at 3 “4 times those of RAID 0. And for high read rates (i.e., read-only tablespaces and other segments used during query-only usage), RAID 5 is the best HA solution. RAID 0+1 is just too expensive per GB; it is two times that of RAID 0. Since most users desire an HA data warehouse solution, the only valid choices are RAID 0+1 and RAID 5. The data warehousing DBA will have to decide whether disk space or loading time is in shorter supply. Assuming that a data warehouse will require 1.5 terabytes of usable disk space, Table 3-3 shows just how many disks are needed for typical RAID 0+1 and RAID 5 solutions. Table 3-3. Example of RAID Selection vs. Usable Disk Space
The key issue is the physical number of drives required. Disk array vendors sell different cabinet and cache sizes based on the physical number of drives to be housed. Obviously, the bigger cabinets and caches that handle larger disk farms cost more money. You can also use multiple disk arrays, assuming that you prefer more, smaller drives instead of 18GB or 36GB drives. The RAID 0+1 solution would then consume 334 9GB drives. There are few, if any disk array cabinets that size, so you'd have to buy multiple disk arrays. That sounds easy, but remember that would require buying very expensive high-speed memory for each disk array. Moreover, it would require splitting your database server's I/O channels across those disk arrays, thereby reducing your overall I/O bandwidth per database server (i.e., each database server has: No. of Controllers/No. of Disk Arrays in Bandwidth per Disk Array ). There's one final and key aspect to successfully implementing RAID 0+1 and RAID 5 disk configurations: choosing the optimal stripe size and stripe set size. This is probably the least understood and most debated aspect of RAID for data warehousing. Striping is simply the process of writing data across multiple disk drives to maximize the number of drives handling each I/O request. It's another "divide and conquer" technique. The idea is that N drives working together should be able to complete an I/O request quicker than one drive. The stripe size is the number of bytes written to a disk before switching to the next disk in the same group, and the stripe set is the number of disks from that group . Figure 3-8 shows an example of RAID 0+1 with a stripe size of 64 kilobytes (K) and a stripe set size of 4. That means that four disks form a group where every 64K is on a different disk. Bytes 1 “64 are on Disk 1, Bytes 65 “128 are on Disk 2, and so on. If an I/O request is larger than 256K (i.e., Stripe Size * Stripe Set Size ), then that I/O request wraps back around to Disk 1 and repeats the entire process, which can be a performance problem. Look again at Figure 3-8 and assume we need to write 512K to a table whose Oracle data blocks all are within the same data file. We'll be asking the disk array to perform eight concurrent 64K writes to the same stripe set, or two 64K writes per physical disk. We would have been much better off if the stripe set size had been eight drives or if the data had been spread across two data files that fell into two different stripe sets, so that eight different physical disk drives could have handled the eight concurrent write requests . Figure 3-8. Example RAID 0+1 Striping Parameters Translation: The data warehousing DBA must very carefully consider the database block size and multi-block initialization parameter settings when choosing the stripe size and stripe set size. RAID I/O performance can be improved or set back an order of magnitude by the positive or negative interaction of these settings. |
EAN: 2147483647
Pages: 79