Automatic Storage Management for Cluster Management

As you learned in Chapter 4, Oracle 10g now ships its own volume manager with the introduction of Automatic Storage Management, or ASM. ASM was designed to ease the tasks associated with database administration by automatically distributing and rebalancing files across all available raw devices within the ASM disk group. ASM offers a unique alternative for supporting the physical requirements of RAC by combining Oracle-managed files and volume-management technology, all within a shared cluster environment. This new concept offers RAC DBAs the opportunity to create Oracle data files in a self-managing and automatically stripped and mirrored clustered environment. By using ASM with RAC, DBAs can eliminate the manageability issues that come with raw devices as well as the extra layer of software required for a cluster file system.

Managing ASM with 10g RAC

In order to utilize ASM volume management for 10g RAC, each node within your cluster must have its own ASM instance. If you use the 10g DBCA utility to create your RAC database, DBCA will also provide you the opportunity to create your ASM instance.

Because ASM is designed specially for the Oracle database, only Oracle-specific files are supported. All other files can only be utilized locally on each node. Files supported with ASM include the following:

• Database files

• Control files

• Online redo log files

• Archived redo log files

• Flash recovery area files

• RMAN image copy and backupset files

Files not supported with ASM include the following:

• ORACLE_HOME install files

• CRS_HOME install files

• ORACLE_BASE files, including alert log, trace files, and so on

• CRS voting disk and OCR files

• Oracle 9i external table files

• Output data from UTL_FILE

• Any user applicationspecific files, such as XML or Java files

ASM Maximum File Size Limitation with 10g RAC

For files that are supported on ASM disk groups, Oracle does impose specific maximum file sizes depending on the redundancy level of each ASM disk group. These maximum file sizes are enforced regardless of whether you create your 10g tablespaces with the small file or BigFile syntax. Because the data file size limit essentially depends on the DB_BLOCK_SIZE of your database, the ASM limitations would only affect 32KB block size for small file tablespaces and all block sizes using BigFile tablespaces.

The maximum file sizes per redundancy level are as follows:

• External redundancy: 300GB

• Normal redundancy: 150GB

• High redundancy: 100GB

If you created your tablespace data files using normal syntax or specifically using the small file syntax, the maximum file sizes on ASM disk groups would be as follows:

• 4KB block size: 16GB

• 8KB block size: 32GB

• 16KB block size: 64GB

• 32KB block size: 100GB if on high redundancy. However, if on normal or external redundancy, the normal limit of 128GB will apply.

If you created your tablespace data files using the new BigFile syntax, then the ASM maximum file limits will apply to all block sizes:

• 2KB block size: Normally 8TB, but restricted by ASM limits

• 4KB block size: Normally 16TB, but restricted by ASM limits

• 8KB block size: Normally 32TB, but restricted by ASM limits

• 16KB block size: Normally 64TB, but restricted by ASM limits

• 32KB block size: Normally 128TB, but restricted by ASM limits

Because ASM is the new volume management/file system solution that is designed specifically for Oracle database files, it is now the recommended shared storage environment for all 10g RAC environments. Also, ASM does not require any additional license fee as you would find with third-party cluster vendors. If you are interested in migrating your clustered environment to ASM, please refer to the section "Non-ASM to ASM Migration" in Chapter 4.

Possible Veritas Library Issue for 10g

As noted on MetaLink, Veritas 3.5 and 4.0 are certified with 9iR2 32bit RAC and Veritas 4.1 is certified with 9i and 10g 64bit RAC. However, during RAC installation, the Veritas library file called libskgxp9.so is overwritten with Oracle's libskgxpu.so library file, which is not the correct library file for Veritas CFS support. Proof of this can be found in your alert log if you see a message that reads cluster interconnect IPC version: Oracle UPD/IP. To properly link the correct library files, you will need to perform the following steps:

For Veritas 3.5 or 4.0 using 32bit 9i RAC:

1.	cd /opt/ORCLcluster/lib/9iR2
2.	cp libskgxn2_32.so ../libskgxn2.so
3.	cp /opt/ORCLcluster/lib/9iR2/ libskgxp92_32.so $ORACLE_HOME/ lib/libskgxp9.so
4.	cp /opt/ORCLcluster/lib/9iR2/ libskgxp92_32.so $ORACLE_HOME/ lib/libskgxpu.so
5.	cd $ORACLE_HOME/lib
6.	ln s /usr/lib/libodm.so libodm9.so
7.	$ORACLE_HOME/rdbms/lib/make f ins_rdbms.mk rac_on ioracle or run 'relink all' to relink the oracle libraries

For Veritas 4.1 using 64bit 10g and 9i RAC:

1.	cd /opt/VRTSvcs/rac/lib
2.	cp libskgxp10_64.so $ORACLE_HOME/lib/libskgxp10.so
3.	cd $ORACLE_HOME/lib
4.	mv libodm10.so libodm10.so.old
5.	ln s /usr/lib/sparcv9/libodm.so libodm10.so
6.	relink libraries if necessary

If the correct library is linked with Veritas, you should see the following from your alert log: cluster interconnect IPC version: Veritas IPC 4.1.