Oracle 10g on RHEL4,U3 AMD64

From Olden Timey Wiki

Contents

[edit]

Introduction

Welcome to my notes on installing Oracle 10.2 Enterprise under RedHat Enterprise Linux ES Version 4 on a Rackable Systems AMD64 Cluster.

While these notes are pertinent to the 64bit environment, most of the information here is also applicable to 32bit Oracle/Linux.

There are many many steps to preparing a system for the installation of Oracle. This is not a complete guide, please refer to the links section below for more information.


Resource Limits

You must make some changes to your environment to allocate the resources required of the database server.

If you maintain an environment with multiple Oracle unix users you may find it easier to configure resource limits on a per-group basis rather than user by user.

Simply create a dba group (groupadd dba) and assign that group to all of your Oracle logins (usermod -g dba <each_oracle_user>).

If you choose to implement resource limits on a per-group basis your /etc/security/limits.conf will look something like this: 

@dba            soft    memlock         3145728
@dba            hard    memlock         3145728

@dba            soft    nofile          65536
@dba            hard    nofile          65536 

@dba            soft    nproc           16384
@dba            hard    nproc           16384

@dba            soft    stack           32768


PAM

Be sure to modify /etc/pam.d/login to include the pam_limits module or your changes above will not be configured. 

## DBA: Oracle 10G needs this!
session    required     pam_limits.so


SSHD

The secure shell daemon must also be configured to call login(1), and privilege separation should be turned off. 


###############################################################################
## Oracle 10g on AMD64 needs these to setup /etc/security/limits.conf
## cp /usr/local/etc/sshd_conf /etc/ssh/sshd_config
###############################################################################
UseLogin yes
UsePrivilegeSeparation no


Kernel Settings 

###############################################################################
## Oracle resource settings
## cp /usr/local/etc/sysctl.conf.10g /etc/sysctl.conf 
###############################################################################
fs.file-max = 65536
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.ip_local_port_range=1024 65000
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
kernel.shmall = 2097152
kernel.shmmax = 8589934592
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128

DNS

/etc/hosts must contain entries for public,private and vip interfaces of both boxes. 

bash# hostname
box3

# grep box3 /etc/hosts
192.168.2.X box3 box2.domain.com

cat /usr/local/etc/hosts >>/etc/hosts

{notes about hostname being properly defined in /etc/hosts}

[edit]

Verification

Oracle provides a verification utility which provides very useful in determining the state of your system.

Download the 10gr2_rdbms_linuxamd64_hcve_043006.tar package (or the package for your architecture) from Oracle. {Link?}

Next unpack the file (tar xvf 10gr2_rdbms_linuxamd64_hcve_043006.tar), change directory to the package directory and run the validator with the supplied .txt file as its only argument. 

[orauser@yourbox]$ perl validate.pl 10gr2_rdbms_linuxamd64_hcve_043006.txt

(Output truncated)

Test Results
15:57, 18 Jul 2006 (EDT)15:57, 18 Jul 2006 (EDT)

  ID NAME                 RESULT C VALUE
===== ==================== ====== = ========================================
  10 OS certified?        PASSED = Certified with 10gR2 RDBMS
  20 User in /etc/passwd? PASSED = userOK
  30 Group in /etc/group? PASSED = GroupOK
  40 Input ORACLE_HOME    RECORD   $ORACLE_HOME
  50 ORACLE_HOME valid?   FAILED = OHnotvalid
  60 O_H perms OK?        FAILED = OHNotExist WrongPerms
  70 Umask set to 022?    PASSED = UmaskOK
  80 LDLIBRARYPATH unset? PASSED = UnSet
  90 JAVA_HOME unset?     PASSED = UnSet
 100 Other O_Hs in PATH?  PASSED = NoneFound
 110 oraInventory perms   PASSED = oraInventoryNotFound
 120 /tmp adequate?       PASSED = TempSpaceOK
 130 Swap (in Mb)         PASSED > 4094
 140 RAM (in Mb)          PASSED > 16040
 150 Swap OK?             FAILED = SwapLessThanRAM
 160 Disk Space OK?       FAILED = OHNotValid
 170 Kernel params OK?    PASSED = KernelOK
 180 Got ld,nm,ar,make?   PASSED = ld_nm_ar_make_found
 190 ulimits OK?          PASSED = ulimitOK
 204 RHEL3 rpms(Pt1) ok?  PASSED = NotRHEL3
 205 RHEL3 rpms(Pt2) ok?  PASSED = NotRHEL3
 206 RHEL4 Pt 1 rpms ok?  FAILED = binutils-2.15.92.0.2-18 RHEL4rpmsPart..>
 207 RHEL4 Pt 2 rpms ok?  PASSED = RHEL4rpmsPart2OK
 209 ip_local_port_range  PASSED = ip_local_port_rangeOK
[edit]

Packages

rpm -e compat-libstdc++-296-2.96-132.7.2
rpm -i cvuqdisk-1.0.1-1.rpm
rpm -i j2sdk-1_4_2_09-linux-i586.rpm
rpm -i /mnt/rhel/rhes3_x86/RedHat/RPMS/compat-gcc-c++-7.3-2.96.128.i386.rpm
rpm -i /mnt/rhel/rhes3_x86/RedHat/RPMS/compat-libstdc++-devel-7.3-2.96.128.i386.rpm
rpm -i /mnt/rhel/rhes3_x86/RedHat/RPMS/compat-libstdc++-7.3-2.96.128.i386.rpm

(About binutils pkg)


[edit]

CRS

CRS (Cluster Ready Services) are...


Files and Directories

Reminder: crs must go into a unique directory $ORACLE_HOME/crs

  • /etc/oracle
  • /opt/oracle


Interfaces

Oracle 10g servers require THREE distinct network addresses:

  • private: cluster interconnect interface
  • public: 'normal' ethernet interface
  • virtual: virtual interface of the oracle cluster


Private Interface

The private interface is the backend or 'interconnect' link used by the nodes to communicate among each other. This is often a 100/1000MB private network.


Public Interface

This is the 'normal' ethernet interface used by the node for non-database communications. (ie: mail, login, http, etc)


Virtual Interface

Oracle 10g provides network failover functionality; when a cluster node becomes unreachable another node in the cluster can assume the virtual ethernet interface of the failed node.

The virtual interface address must reside on the same network as the 'public' interface.

For example, if the public address for your node is 192.168.1.100 you'd need to choose an available address on the 192.168.1.0/24 network for your virtual interface.

The virtual interface on a cluster node MUST BE A AVAILABLE on the network and MUST NOT BE PRECONFIGURED BY THE OPERATING SYSTEM. The cluster manages the aliases itself independent of the local operating system.

The ip addresses of all interfaces should be defined in every local /etc/hosts file. For example: 

## public
192.168.2.67    node1 node1.example.org
192.168.2.68    node2 node2.example.org
## private
10.0.0.100      node1-i node1-i.example.org 
10.0.0.101      node2-i node2-i.example.org
## virtual
192.168.2.169   node1-v node1-v.example.org
192.168.2.170   node2-v node2-v.example.org

Authentication

{What you need}

Any stray output from ssh will confuse the life out of the CRS installer; it will report 'connection refused' and a number of other strange errors which do not exist.

Text from /etc/issue, /etc/motd or even ssh itself will halt the install process.

For example: 

Warning: No xauth data; using fake authentication data for X11 forwarding.


You can fix this by disabling x11 forwarding on the client: 

/etc/ssh/ssh_config

Host *
  ForwardAgent no
  ForwardX11 no
  CheckHostIP no
  StrictHostKeyChecking no


Workarounds

Oracle Notification Server Configuration Assistant Failure (Search)

Make certain that you zero the crs voting and registry files before re-installing crs!

You should also delete your /etc/oracle/ocr.loc file as well . The settings stored in this file may confuse the installer.

RACGVIP Errors installing Oracle

Should you get the 'asflasdfasdf' error while re-installing oracle you should suspend OUI and execute 'racgvip' as root . Make certain to have your $DISPLAY variable properly set to YOUR_DESKTOP_IP:DISPLAYNO, and an X11 server available on your desktop. 

bash# export DISPLAY=mydesktop_ip_or_dns.name.com:0
bash# $CRS_HOME/bin/racgvip
[edit]

Notes

Reminders

  • Create oracle user with primary group oinstall
  • Make sure $ORACLE_HOME is writable by oracle user
  • LD_LIBRARY_PATH must be unset
  • remove ulimit calls from .bash_profile


Adding swap space

Add 1G of temporary swap space (change count=to number of MB desired): 

dd if=/dev/zero of=/path/temporary_swap bs=1024k count=1000
mkswap /path/temporary_oracle_swap
swapon /path/temporary_oracle_swap

ssh clients should disable x11 forwarding via /etc/ssh/ssh_config:

[edit]

Links

Oracle 10g RAC installation on Red Hat Linux 3.0 EL - part 1

Oracle 10g RAC installation on Red Hat Linux 3.0 EL - part 2

10g RAC: Troubleshooting CRS Reboots (Metalink: Note:265769.1)

Oracle Clusterware consolidated logging in 10gR2 (Metalink: Note:265769.1)

10g RAC: Troubleshooting CRS Root.sh Problems (Metalink: Note:240001.1)

CVU FAQ (Metalink: Note:316817.1)

CRS and 10g Real Application Clusters (Metalink: Note:259301.1)

How to Configure Virtual IPs for 10g RAC (Metalink: Note:264847.1)

Oracle® Database Installation Guide 10g Release 2 (10.2) for Linux x86-64

CLUSTER VERIFICATION UTILITY FAQ

Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 on AMD64/EM64T

Pre-Install checks for 10gR2 RDBMS (10.2.x) - Linux AMD64/EM64T Platforms

Installing 10.2.0.1 Db On A Redhat Linux X86-64 Os Version 4.0, Errors

Oracle® Database Installation Guide 10g Release 2 (10.2) for Linux x86

Retrieved from "http://wiki.oldtimeynerd.net/index.php/Oracle_10g_on_RHEL4%2CU3_AMD64"