We are trying to find the best tuning options for haproxy for get and post request that come from a client (not users browsing the web type of deal).
Running a jmeter test with 30k threads that consists of 5 calls to the servers, 1 user reg, and a few update calls. These push json data though the pipeline.
Here us our current config for haproxy
global
log /dev/log local0 #notice
maxconn 14000
tune.bufsize 128000
user netcom
group netcom
pidfile /tmp/haproxy.pid
daemon
nbproc 7
#debug
#quiet
defaults
log global
mode http
### Options ###
option httplog
#option logasap
option dontlog-normal
#option dontlognull
option redispatch
option httpchk GET /?method=echo HTTP/1.1
option tcp-smart-accept
option tcp-smart-connect
option http-server-close
#option httpclose
#option forceclose
### load balance strategy ###
balance leastconn
#balance roundrobin
### Other ###
retries 5
maxconn 14000
backlog 100000
### Timeouts ###
#timeout client 25s
timeout client 60s
#timeout connect 5s
timeout connect 60s
#timeout server 25s
timeout server 60s
timeout tunnel 3600s
timeout http-keep-alive 1s
#timeout http-request 15s
timeout http-request 60s
#timeout queue 30s
timeout queue 30s
timeout tarpit 60s
listen stats *:1212
stats enable
stats show-node
stats show-desc xxxxProxy
stats realm xxxxProxy\ Statistics
stats auth xxxx:xxxx
stats refresh 5s
stats uri /
frontend http-in
bind *:1111
bind *:2222 ssl crt /home/netcom/nas/haproxy/xxxx.co.pem verify optional
acl user_request url_reg method=user.register
use_backend user_group if user_request
default_backend other_group
backend user_group
server n15 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n2 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n9 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n14 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n22 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n24 xxxx:8080 maxconn 3500 check port 8097 inter 2000
server n25 xxxx:8080 maxconn 3500 check port 8097 inter 2000
and our sysctl on centOS 6
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_recycle = 1
net.core.wmem_max = 12582912
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 20480 174760 25165824
net.ipv4.tcp_wmem = 20480 174760 25165824
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.core.netdev_max_backlog = 10000
# Syn flood
net.ipv4.tcp_max_syn_backlog = 8096
net.core.somaxconn = 8096
anyone point out any blaring issues that they can see off the top of your head. Unfortunately I do not have the expertise in haproxy so looking for help from the community.
What I also prob need to figure out is how to find the max connections the box can handle, its on 1 gig network and all the backends are on one gig as well. Here is screen shot from the haproxy admin http://grab.by/r12c, note we are running it with more than one core so this is a snapshot of the one core.. since the web admin as far as I can tell cant show everything.. any idea how to get the max conn that haproxy is getting from cmd line?
anyhow just working though this and hope that anyone can give some tips or pointers.
Well the first thing is that it doesn't seem like you should be running multiple processes of haproxy. Typically you won't want to do that, especially because you are busy testing and trying to see the maxconn's. On a single core haproxy can way outperform the maxconn setting you have anyway.
I went through Snapt's sysctl's and you have most of the; I noticed it's also adding these --
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
Also, leastconn is not going to be worthwhile I would suggest roundrobin. Because you are doing HTTP traffic which consists of many small requests (I guess that depends though to be honest). These are such minor things though.
来源：https://stackoverflow.com/questions/19282514/haproxy-tuning-for-performance