Using HAProxy with MQTT lowers the number of max connections [Connection lost error]

Question

I am trying to establish 10k client connections(potentially 100k) with my 2 MQTT brokers using HAProxy as a load balancer.

I have a working simulator(using Java Paho libr