::Oracle 10 RAC – RHEL 4 Linux on NFS tuning. ::

It has been a couple of years since I posted anything along the lines of the Oracle/linux tuning I did. Here is the system tuning for an Oracle 10 RAC/RHEL 4 2.6/NFS tuning that will get you 10% better performance than the oracle recommended settings.

To give some background, I am the systems architect for a fortune 100 company. I have done the system setup and tuning for our linux environment. I consulted at Oracle corporate and revised their Solaris environment several years ago. Today I am architecting the Linux environment.

Below are the sysctl.conf parameters for a Linux opteron setup using NFS storage. Oracle has some great tuning sites now, but we did systematic and specific testing. We get near wire speed using oracle’s directio in Linux. Three Oracle RAC servers with dual bonded interfaces over two networks connected to two Network Appliance filers with trunked and etherchanneled interfaces get over 400MB/s sustained traffic in our Oracle based Data Warehouse.

Your shared segment values may vary, but we don’t find a variance in the testing as long as yours isn’t larger than 21GB you should be fine with these settings, linux does not allocate more than you request but limits it to this value. Also HUGETLB did not provide any positive effect, only negative.

Additionally we tested with and with out numa, while there are some relevances to poorly balanced memory over the 4 cpus without setting numa=off, we don’t really see a performance impact by leaving it so.

Soooo just do the normal thing. We also see no swap. These systems are 4 cpu 32GB memory. Tuned out so we don’t swap.

# Oracle RAC Params
kernel.sem = 250 256000 100 1024
# this is set for a 21GB SGA/shared.
kernel.shmmax = 21474836480
kernel.shmall = 21474836480
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216
sunrpc.tcp_slot_table_entries = 128

# For NFS since DB is on NetApp
net.ipv4.ipfrag_low_thresh = 262144
net.ipv4.ipfrag_high_thresh = 393216

# Net tuning
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216


vm.page-cluster = 40
vm.lower_zone_protection = 100
# Reference for min_free_kbytes
# http://people.redhat.com/dduval/kernel/min_free_kbytes.html
vm.min_free_kbytes = 200000
vm.swappiness = 0
vm.dirty_background_ratio = 10
vm.dirty_expire_centisecs = 1000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 100
vm.vfs_cache_pressure = 40

Leave a Reply