Ran into an obscure tcp bug/behavior the last few days on my AX2000 loadbalancer. By default the layer4 tcp loadbalancing mechanism decouples the tcp connections such that you have two separate ones: (client -> a10) + (a10 -> server). The backend (a10 -> server) has a timeout default of 120 seconds, but for some reason the session table only updates every minute so it is in effect a 180 second timeout (it will still work and show up in a “show session” command with an Age of 0 for the last minute).
client -> a10 has no such timeout (or its much longer), so if you have something with a long idle connection (like say, mysql_pconnect on a development webserver) you can easily run into a case where the front end tcp socket considers itself in an established state while the backend one has timed out and disappeared. The next packet through that connection will need a new backend tcp connection to be established. Only the A10 doesn’t do this. It just blackholes the traffic. No TCP RST, no automatically firing up a new connection, just … fail.
As a workaround you can apply a custom tcp template that will properly return a TCP RST. Search your cli reference PDF for “reset-rev” for info. In my mind this is somewhat contradictory to the whole notion of connection-pooling/connection-reuse, but I’m not sure if mysql would even allow the new socket to just reappear since it probably already cleaned up the session.