Redhat Enterprise Oracle Large SGA

From Olden Timey Wiki

This article documents some of the finer points of increasing the Oracle SGA under RedHat Enterprise Linux 3. These notes are not a replacement for understanding what you are doing. Please consult your local Oracle guru should you have any questions.

Oracle Documentation

Metalink doc notes: 262004.1, 260152.1, 510334.994 .

Oracle Patch

Install 3566570 for Oracle 9.2.0.5 to add hugetlb and remap_file_pages support .

Should opatch fail, perform the following:

cd /tmp
unzip p3566570_9205_LINUX.zip
cd /opt/oracle/product/9.2/lib
ar rv libgeneric9.a /tmp/3600333/files/lib/libgeneric9.a/sskgm.o
ar rv libgeneric9.a /tmp/3600333/files/lib/libgeneric9.a/sskgmvlm.o
cd /opt/oracle/product/9.2/rdbms/lib
make -f ins_rdbms.mk ioracle

Kernel Choice

The RHEL3/U3 SMP kernels are appropriate for the amount of shared memory we're dealing with (ie: under 16gb or so)

Settings

/etc/rc.d/rc.local

mount -t ramfs ramfs /dev/shm
chown oracle.oracle /dev/shm

/etc/sysctl.conf

## the size of the ramfs chunk is determined by these two settings
kernel.shmmax = 3700000000
vm.hugetlb_pool = 2048 (blocks of TLB to allocate, in 2048k pages)

/etc/security/limits.conf

## Required for Oracle Large SGA
oracle soft memlock 3145728
oracle hard memlock 3145728

/etc/init.d/sshd (within the start() function)

ulimit -l 3145728

/etc/ssh/sshd_config

UseLogin yes

Shared Memory Calculations

To determine the size of the shared memory segment:

- Reboot with hugepages settings and make sure via /proc/meminfo that the amount of requested memory was actually allocated..

- Configure init.ora

## To support Oracle large SGA
use_indirect_data_buffers=true
db_block_buffers=246083 (how many db_block_size buffers to allocate)

- Restart Oracle

- Run ipcs(8) to determine the size of the shared memory segment allocated(see metalink doc)

# ipcs

------ Shared Memory Segments --------

key shmid owner perms bytes nattch status

0x00000000 1572864 oracle 660 2141192192 11

Convert to MB: 2141192192 bytes = 2042 MB and add 4mb to this number.

Then, store this value in sysctl.conf vm.hugetlb_pool variable.

Determining HugeTLB allocation

# tail -3 /proc/meminfo
HugePages_Total:  1000
HugePages_Free:   1000
Hugepagesize:     2048 kB

Relinking Oracle

Modify shmmax:

% echo 3000000000 > /proc/sys/kernel/shmmax

Relocate the SGA:

% cd $ORACLE_HOME/rdbms/lib
% cp ksms.s ksms.s_orig
% genksms -s 0x15000000 > ksms.s
% make -f ins_rdbms.mk ksms.o
% make -f ins_rdbms.mk ioracle 

Explanation of Relocating SGA

% cp ksms.s ksms.s_orig
% genksms -s 0x15000000 > ksms.s (run util to re-generate the assembly code describing memory relationship, lowering the base sga address )
% make -f ins_rdbms.mk ksms.o (rebuild ksms.o via ins_rdbms.mk makefile)
% make -f ins_rdbms.mk ioracle (relink oracle binary via ins_rdbms.mk makefile)

Potential Bugs

Subject: Hugetlb patch (patch number 3600333) breaks sqlplus from cron.