Oracle 8i | Oracle 9i | Oracle 10g | Oracle 11g | Miscellaneous | Oracle RAC | Oracle Apps | Linux

Oracle9i (9.2.0.4.0) RAC Installation on Tru64 5.1b

This article is intended as a brief guide to installing Oracle9i (9.2.0.3.0) Real Application Clusters (RAC) on Tru64 5.1b. It assumes that the servers are using a cluster file system and all hardware cluster setup has been completed:

Download Software

Download Sun's Java Development Kit (JDK 1.3.1).

Download the Oracle installation files from otn.oracle.com.

Unpack Files

First unzip the files:
gunzip Tru64_9201_Disk1.cpio.gz
gunzip Tru64_9201_Disk2.cpio.gz
gunzip Tru64_9201_Disk3.cpio.gz
gunzip Tru64_9201_Disk4.cpio.gz
Next unpack the contents of the files:
cpio -idcmv < Tru64_9201_Disk1.cpio
cpio -idcmv < Tru64_9201_Disk2.cpio
cpio -idcmv < Tru64_9201_Disk3.cpio
cpio -idcmv < Tru64_9201_Disk4.cpio
You should now have four directories (Disk1, Disk2, Disk3 and Disk4) containing installation files.

Set Kernel Parameters

or a complete list of kernel settings read Oracle 9i Real Application Cluster (RAC) on Compaq Tru64 UNIX.

The current settings can be viewed using commands like:
/sbin/sysconfig -q ipc
/sbin/sysconfig -q proc
If any settings need to be altered they should be added to a file named sys_tuning.stanza and applied with the sysconfigdb command. The typical alterations are:
vm:
  new_wire_method = 1 # Set to 0 if <= 5.1b PK2
  vm_bigpg_enabled = 0

ipc:
  shm_max = 4278190080 # (4 GB less 16MB)
  shm_min = 1
  shm_mni = 256
  shm_seg = 128

inet:
  udp_sendspace = 65536
  udp_recvspace = 65536

proc:
  max_per_proc_stack_size = 33554432
  per_proc_stack_size = 33554432
  per_proc_data_size = 335544320
     
vfs:
  fifo_do_adaptive = 0
Check Metalink Note: 169706.1 to confirm the current best settings.

With the file created it can be applied by issuing the following command on each node:
/sbin/sysconfigdb -m -f sys_tuning.stanza
Once the file is applied check for, and remove, any duplicate or conflicting entries (like vm_swap_eager) in the /etc/sysconfigtab file. The server must be rebooted to take on the new settings. This can be done using the following command:
shutdown -r now
The performance of gettimeofday() can be improved in Oracle by performing the following as root on each node:
mknod /dev/timedev c 15 0
chmod 664 /dev/timedev

Setup

Install the Java development kit:
tar -xvjf your_JDK_version -C /usr/opt/
Create the new groups and users. Since the environment is clustered this only needs to be done on a single node:
# addgroup
Enter a new group name or <Return> to exit: oinstall
Enter a new group number [200]: <CR>
Group oinstall was added to the /etc/group file.

# addgroup
Enter a new group name or <Return> to exit: dba
Enter a new group number [201]: <CR>
Group dba was added to the /etc/group file.

# addgroup
Enter a new group name or <Return> to exit: apache
Enter a new group number [202]: <CR>
Group apache was added to the /etc/group file.


# adduser
Enter a login name for the new user (for example, john): oracle
Enter a UID for (oracle) [101]: 400
Enter a full name for (oracle): Oracle software owner & admin
Enter a login group for (oracle) [users]: oinstall

Enter another group that (oracle) should be a member of.
(<Return> only if none): dba

Enter another group that (oracle) should be a member of.
(<Return> only if none):
Enter a parent directory for (oracle) [/usr/users]: <CR>
The shells are:

/usr/bin/sh       /usr/bin/ksh      /bin/sh           /bin/ksh
/usr/bin/csh      /usr/bin/posix/sh /bin/csh

Enter a login shell for (oracle) [/bin/sh]: /bin/ksh

Adding new user...
You must enter a new password for (oracle).
Changing password for oracle.

New password:
Retype new password:
Hashed database not in use, only /etc/passwd text file updated.

Finished adding user account for (oracle).

# adduser
Enter a login name for the new user (for example, john): apache
Enter a UID for (apache) [401]: <CR>
Enter a full name for (apache): APACHE Account
Enter a login group for (apache) [users]: oinstall

Enter another group that (apache) should be a member of.
(<Return> only if none): apache

Enter another group that (apache) should be a member of.
(<Return> only if none):
Enter a parent directory for (apache) [/usr/users]: <CR>
The shells are:

/usr/bin/sh       /usr/bin/ksh      /bin/sh           /bin/ksh
/usr/bin/csh      /usr/bin/posix/sh /bin/csh

Enter a login shell for (apache) [/bin/sh]: /bin/ksh

Adding new user...
You must enter a new password for (apache).
Changing password for apache.

New password:
Retype new password:
Hashed database not in use, only /etc/passwd text file updated.

Finished adding user account for (apache).
Create additional directories for RAC configuration files:
mkdir /var/opt/oracle
chown oracle:oinstall /var/opt/oracle
chmod 755 /var/opt/oracle

mkdir /usr/local/bin
chmod 755 /usr/local/bin
Append the following line to the .profile file under the apache home directory (/usr/users/apache):
PATH=$PATH:/usr/local/bin; export PATH
Append the following line to the /etc/services file:
listener 1521/tcp # Oracle
Change the ownership of some filesystem directories:
chown -R oracle:oinstall /u01
chown -R oracle:oinstall /u02
Append the following lines to the .profile file in the oracle users home directory (/usr/users/oracle):
TMP=/u01/temp; export TMP
TMPDIR=$TMP; export TMPDIR

# Oracle RAC
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/9.2.0.1.0; export ORACLE_HOME
PATH=.:$PATH:/usr/sbin:/usr/ccs/bin:/usr/bin:/usr/bin/X11:/usr/local/bin
PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/Apache/Apache/bin:/usr/opt/java131:/usr/opt/java131/bin; export PATH
member=`clu_get_info|grep "this member"|awk '{print $6}'`; export member
ORACLE_SID=db9i$member; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
SRVM_SHARED_CONFIG=/u01/db9i_configuration; export SRVM_SHARED_CONFIG

DISPLAY=localhost:0.0; export DISPLAY

stty erase "^H"
set -o vi
set filec
Make the Oracle directories:
mkdir /u01/tmp
mkdir -p /u01/oradata/db9i
mkdir -p /u02/oradata/db9i
As we are using system wide clusterware we don't need to configure individual partitions. We must simply create the configuration file:
touch /u01/db9i_configuration
Login as root and issue the following command:
xhost +<machine-name>

Installation

Log into the oracle user. If you are using X emulation then set the DISPLAY environmental variable:
DISPLAY=<machine-name>:0.0; export DISPLAY
Start the Oracle Universal Installer (OUI) by issuing the following command in the Disk1 directory:
./runInstaller
With the exception of the cluster node selection screen the installation follows the usual format. When using a clusterwide configuration leave only the default node selected. Selecting multiple nodes may cause locking problems within the Oracle inventory, especially when applying the 9.2.0.4.0 patch. Select a software only installation so the latest patch can be applied before instance creation. When prompted run the root.sh on all nodes in the cluster.

Install 9.2.0.4.0 patch before any instances are created. Make sure only the default node in the cluster is selected during the install.

Some files must be member specific. The mkcdsl command creates context dependent symbolic links so that a single shared directory is actually link to a member-specific directory:
mkcdsl -ac /u01/app/oracle/product/9.2.0.1.0/network/admin
mkcdsl -ac /u01/app/oracle/product/9.2.0.1.0/network/agent
mkcdsl -ac /u01/app/oracle/product/9.2.0.1.0/network/log
mkcdsl -ac /u01/app/oracle/product/9.2.0.1.0/Apache/Apache/conf
If for some reason you need to remove a CDSL simply remove the link directory and remove it from the inventory: 
rm /u01/app/oracle/product/9.2.0.1.0/Apache/Apache/conf
mkcdsl -i /u01/app/oracle/product/9.2.0.1.0/Apache/Apache/conf
The physical member-specific directories will still be present so the original data can be copied back into the correct location before the member-specific directories are cleaned up manually.

Next create a $ORACLE_HOME/network/admin/listener.ora file for each member. This can be done using the Oracle Net Configuration Assistant (netca) or by adding the file manually to each node like: 
LISTENER =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = server01)(PORT = 1521))
  )

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/9.2.0.1.0)
      (PROGRAM = extproc)
    )
    (SID_DESC =
      (SID_NAME = db9i1)
      (ORACLE_HOME = /u01/app/oracle/product/9.2.0.1.0)
    )
  )
Remember, the HOST and SID_NAME should reflect the member number. Since the network/admin directory is a CDSL there will actually be one file for each node.

Each node should have a copy of the tnsnames.ora file like:
LISTENER_db9i1 =
  (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = server01)(PORT = 1521))
  )

LISTENER_db9i2 =
  (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = server02)(PORT = 1521))
  )

db9i1 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = server01)(PORT = 1521))
    )
    (CONNECT_DATA =
      (ORACLE_SID = db9i1)
    )
  )

db9i2 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = server02)(PORT = 1521))
    )
    (CONNECT_DATA =
      (ORACLE_SID = db9i2)
    )
  )

db9i =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = server01)(PORT = 1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = server02)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = db9i)
    )
  )
The LISTENER_db9i(x) entries are referenced by the SID.LOCAL_LISTENER and SID.REMOTE_LISTENER parameters in the spfile.

Run the following commands on all nodes to start the GSD daemon and the listener:
$ORACLE_HOME/bin/gsdctl start
$ORACLE_HOME/bin/lsnrctl start
Starting the gsd daemon will take longer on subsequent nodes due to the extra inter-node communication.

Start dbca and select the cluster option. When prompted select all nodes in the cluster and proceed with install as normal. If the network configuration is not correct the DBCA will produce errors towards the end of the instance creation, but will complete successfully, leaving the instances down. If the network configuiration is amended the instances will start successfully.

Once the database instances are built they must be added to the cluster configuarion file to allow them to be managed by the srvctl utility. If this is the first clustered database in the server you must initialize the configuration:
srvconfig -init
Once this is done you can add and remove databases and their respective clustered instances using the srvctl utility:
# Add two-node cluster to config
srvctl add database -d db9i -o /u01/app/oracle/product/9.2.0.1.0
srvctl add instance -d db9i -i  db9i1 -n server01
srvctl add instance -d db9i -i  db9i2 -n server02

# remove instance from cluster
srvctl remove instance -d db9i -i db9i2

# Start and stop cluster
srvctl start database -d db9i
srvctl stop database -d db9i

# Start and stop individual instance
srvctl start instance -d db9i -i db9i1
srvctl stop instance -d db9i -i db9i1

# Get status of whole database or specific instance
srvctl status database -d db9i
srvctl status instance -d db9i -i db9i1

# Get current database config
srvctl config database -d db9i
You can also get information about the current cluster configuarion from any open instance by ussuing the following queries:
CONN sys/password@db9i AS SYSDBA

SELECT instance_number,
       instance_name
FROM   v$instance;

SELECT *
FROM   v$active_instances;

Post Installation

Create a file called /sbin/init.d/oracle containing the following:
#!/bin/sh
#
# change the value of ORACLE_HOME to be correct for your
# installation
ORACLE_HOME=/u01/app/oracle/product/9.2.0.1.0
PATH=${PATH}:$ORACLE_HOME/bin:/usr/sbin
HOST=`hostname`
#
# change the value of ORACLE to the login name of the
# oracle owner at your site
#
ORACLE=oracle
export ORACLE_HOME PATH
#
if [ ! "$2" = "ORA_DB" ] ; then
    rsh $HOST -l $ORACLE /sbin/init.d/oracle $1 ORA_DB
    exit
fi
#
LOG=$ORACLE_HOME/startup.log
touch $LOG
chmod a+r $LOG
#
member=`clu_get_info|grep "this member"|awk '{print $6}'`
#

case $1 in
'start')
        echo "$0: starting up" >> $LOG
        date >> $LOG
        # Start Oracle Net
        if [ -f $ORACLE_HOME/bin/tnslsnr ] ;
        then
                echo "starting Oracle Net listener"
                $ORACLE_HOME/bin/lsnrctl start >> $LOG 2>&1
        fi
        echo "starting Oracle databases"

        # Amended for RAC
        $ORACLE_HOME/bin/gsdctl start
        $ORACLE_HOME/bin/srvctl start instance -d db9i -i db9i$member >> $LOG 2>&1
        $ORACLE_HOME/bin/agentctl start >> $LOG 2>&1
        ;;
'stop')
        echo "$0: shutting down" >> $LOG
        date >> $LOG
        # Stop Oracle Net
        if [ -f $ORACLE_HOME/bin/tnslsnr ] ;
        then
                echo "stopping Oracle Net listener"
                $ORACLE_HOME/bin/lsnrctl stop >> $LOG 2>&1
        fi
        echo "stopping Oracle databases"

        # Amended for RAC
        $ORACLE_HOME/bin/agentctl stop >> $LOG 2>&1
        $ORACLE_HOME/bin/srvctl stop instance -d db9i -i db9i$member >> $LOG 2>&1
        $ORACLE_HOME/bin/gsdctl stop
        ;;
*)
        echo "usage: $0 {start|stop}"
        exit
        ;;
esac
#
exit
Use chmod to set the privileges to 750:
chmod 750 /sbin/init.d/oracle
Add the following entry into the .rhosts file in the oracle user login home directory to allow root login access to the account:
<cluster-alias> root
<node1-name> root
<node2-name> root
Link the file into the startup and shutdown directories using:
ln -s /sbin/init.d/oracle /sbin/rc3.d/S99oracle
ln -s /sbin/init.d/oracle /sbin/rc0.d/K01oracle
Add root user to the dba and oinstall groups (/etc/group) to allow the script to function correctly. The relevant instances should now startup/shutdown automatically at system startup/shutdown.

Comment from Pascal Oegerli at www.edorex.ch - When using an ethernet gigabite card as the cluster interconnection instead of using memory channel the RDG protocol does not work. Therefore, you should relink the kernel to use ipc/udp protocol instead of RDG:
make -f ins_rdbms.mk rac_on ipc_udp ioracle

Client Network Configuration

Client applications can take advantage of RACs load balancing and failover features by adjusting their network parameters. A typical tnsnames.ora entry might look like:
db9i = 
  (DESCRIPTION= 
    (LOAD_BALANCE=ON)
    (FAILOVER=ON)
    (ADDRESS=(PROTOCOL=TCP)(HOST=server01)(PORT=1521))
    (ADDRESS=(PROTOCOL=TCP)(HOST=server02)(PORT=1521)) 
    (CONNECT_DATA=
      (SERVICE_NAME=db9i)
      (FAILOVER_MODE=
        (TYPE=SELECT) 
        (METHOD=BASIC)
        (RETRIES=20)
        (DELAY=15)
      )
    )
  )
Depending on your business needs you may wish to partition your processing across different nodes of the RAC depending on the type of processing like:
db9i_oltp = 
  (DESCRIPTION= 
    (ADDRESS=(PROTOCOL=TCP)(HOST=server01)(PORT=1521))
    (CONNECT_DATA= (SERVICE_NAME=db9i))
  )

db9i_reports = 
  (DESCRIPTION= 
    (ADDRESS=(PROTOCOL=TCP)(HOST=server02)(PORT=1521))
    (CONNECT_DATA= (SERVICE_NAME=db9i))
  )
For further information see:
Hope this helps. Regards Tim...

Back to the Top.