It has been a couple of years since I posted anything along the lines of the Oracle/linux tuning I did. Here is the system tuning for an Oracle 10 RAC/RHEL 4 2.6/NFS tuning that will get you 10% better performance than the oracle recommended settings.
To give some background, I am the systems architect for a fortune 100 company. I have done the system setup and tuning for our linux environment. I consulted at Oracle corporate and revised their Solaris environment several years ago. Today I am architecting the Linux environment.
Below are the sysctl.conf parameters for a Linux opteron setup using NFS storage. Oracle has some great tuning sites now, but we did systematic and specific testing. We get near wire speed using oracle’s directio in Linux. Three Oracle RAC servers with dual bonded interfaces over two networks connected to two Network Appliance filers with trunked and etherchanneled interfaces get over 400MB/s sustained traffic in our Oracle based Data Warehouse.
Your shared segment values may vary, but we don’t find a variance in the testing as long as yours isn’t larger than 21GB you should be fine with these settings, linux does not allocate more than you request but limits it to this value. Also HUGETLB did not provide any positive effect, only negative.
Additionally we tested with and with out numa, while there are some relevances to poorly balanced memory over the 4 cpus without setting numa=off, we don’t really see a performance impact by leaving it so.
Soooo just do the normal thing. We also see no swap. These systems are 4 cpu 32GB memory. Tuned out so we don’t swap.
# Oracle RAC Params kernel.sem = 250 256000 100 1024 # this is set for a 21GB SGA/shared. kernel.shmmax = 21474836480 kernel.shmall = 21474836480 net.ipv4.ip_local_port_range = 1024 65000 net.core.rmem_default = 262144 net.core.rmem_max = 16777216 net.core.wmem_default = 262144 net.core.wmem_max = 16777216 sunrpc.tcp_slot_table_entries = 128 # For NFS since DB is on NetApp net.ipv4.ipfrag_low_thresh = 262144 net.ipv4.ipfrag_high_thresh = 393216 # Net tuning net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 fs.file-max=327679 fs.aio-max-nr=3145728 vm.page-cluster = 40 vm.lower_zone_protection = 100 # Reference for min_free_kbytes # http://people.redhat.com/dduval/kernel/min_free_kbytes.html vm.min_free_kbytes = 200000 vm.swappiness = 0 vm.dirty_background_ratio = 10 vm.dirty_expire_centisecs = 1000 vm.dirty_ratio = 20 vm.dirty_writeback_centisecs = 100 vm.vfs_cache_pressure = 40