TCP四层代理透传客户端真实IP

缘起

最近公司准备自建邮箱，大概十年前折腾过自建邮箱的事情，感觉坑很多，这次体会了下确实很多，特别是现在管局对邮箱端口管控越来越严格的情况下，很多之前能行得通的方案需要修改下。
由于SMTP、POP3、IMAP等协议需要做四层代理且还需要能穿透客户端真实IP，硬件网络层代理软件又不能使用的情况下只能考虑软代理，下面是记录的两种解决方案。

方案

Nginx解决方案

1.注意nginx编译时需要加上stream模块及stream_realip_module模块;一个用来四层负载，一个用来获取客户端真实IP
2.开启透传功能proxy_protocol on，用于将连接信息从请求连接的源传递到请求连接到的目标

具体配置如下

# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.16.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx  --with-stream --with-stream_realip_module

# cat /usr/local/nginx/conf/nginx.conf

user  nginx;
worker_processes  auto;

events {
    worker_connections  10240;
    }
include tcp/*.conf;

# cat /usr/local/nginx/conf/tcp/mail.conf

stream {
    log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log /usr/local/nginx/logs/mail.log proxy;

    upstream mail_http {
        server 192.168.188.88:80 ;
    }
    server {
        listen 80;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_pass mail_web;
    }

    upstream mail_https {
        server 192.168.188.88:443 ;
    }
    server {
        listen 443;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_pass mail_https;
    }

    upstream mail_smtp {
        server 192.168.188.88:25 ;
    }
    server {
        listen 25 ;
        proxy_connect_timeout 10s;
        proxy_timeout 300s;
        proxy_protocol on ;
        proxy_pass mail_smtp;
    }

    upstream mail_smtps {
        server 192.168.188.88:465 ;
    }
    server {
        listen 465;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_smtps;
    }

    upstream mail_pop3 {
        server 192.168.188.88:110 ;
    }
    server {
        listen 110;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_pop3;
    }

    upstream mail_pop3s {
        server 192.168.188.88:995 ;
    }
    server {
        listen 995;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_pop3s;
    }

    upstream mail_imap {
        server 192.168.188.88:143 ;
    }
    server {
        listen 143;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_imap;
    }

    upstream mail_imaps {
        server 192.168.188.88:993 ;
    }
    server {
        listen 993;
        proxy_connect_timeout 10s;
        proxy_timeout 30s;
        proxy_protocol on ;
        proxy_pass mail_imaps;
    }

}

Haproxy解决方案

1.开通tcp 模式 mode tcp
2.开启透传 send-proxy

注意：基本上haproxy参数需结合haproxy -c测试，一条条的来过才能确定最终哪个参数起作用

# haproxy -v
HA-Proxy version 1.5.18 2016/05/10
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>

# cat /etc/haproxy/haproxy.cf
listen stats
 mode http
 bind 0.0.0.0:9999
 stats enable
 log global
 stats uri     /haproxy-status
 stats auth    haadmin:yourpassword

listen WEB
    bind 80
    mode http
    server web01 192.168.188.88:80  check 

listen WEBSSL
    bind 443
    mode tcp
    server webssl01 192.168.188.88:443  check 

listen MAIL
    bind 25
    mode tcp
    server mail01 192.168.188.88:25  check send-proxy

listen MAILS
    bind 465
    mode tcp
    server mailssl01 192.168.188.88:465  check send-proxy

listen POP3
    bind 110
    mode tcp
    server pop301 192.168.188.88:110  check send-proxy

listen POP3S
    bind 995 
    mode tcp
    server pop3ssl01 192.168.188.88:995  check send-proxy

listen IMAP
    bind 143
    mode tcp
    server imap01 192.168.188.88:143  check send-proxy

listen IMAPS
    bind 993
    mode tcp
    server imapssl01 192.168.188.88:993  check send-proxy

尾声

当然了，这只是自建邮件服务器其中一个小小的问题，基本上一步一个问题，所以现在自建邮箱越来越少了，基本上都被云邮箱取代了，就连Postfix 2012年的时候都停止更新维护了，估计再后来懂这门手艺的人会越来越少了。

来源：oschina

链接：https://my.oschina.net/u/4313521/blog/4922179

标签

HAproxy

nginx

imap

postfix