:: Linux Tuning :: Large HPC System Tuning

We Run this on our HPC nodes which are 256GB memory, but we also use the same tuneables on 16GB and higher systems. These systems are 10Gb connected and do a great deal of NFS traffic.

Again, turn off tcp offload functions on your ethernet cards.

# Tuning from HPC nodes
net.ipv4.ipfrag_low_thresh = 262144
net.ipv4.ipfrag_high_thresh = 393216
sunrpc.tcp_slot_table_entries = 128

kernel.shmall = 20971520
kernel.sem = 250 32000 100 128
fs.aio-max-nr=3145728
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 4194304
net.core.rmem_max = 16777216
net.core.wmem_default = 4194304
net.core.wmem_max = 16777216
net.core.netdev_max_backlog=3000
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

vm.page-cluster = 4000
vm.pagecache = 90
vm.min_free_kbytes = 200000
vm.swappiness = 0
vm.dirty_background_ratio = 10
vm.dirty_expire_centisecs = 4000
vm.dirty_ratio = 30
vm.dirty_writeback_centisecs = 1500
vm.vfs_cache_pressure = 10000