In my application I have two servers (private and external) and there is a need for the private server to run a script on the external server.
Using ssh, I am trying to test the limits of how many connections it will hold:
while (1) foreach k (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ) ssh -i /path/to/key user@host "ls;sleep 0.3" & sleep 0.06 # <=16.6 times/sec end end
The external server (2 core CPU, 4 GB ram), manage to maintain the incoming connections when they are at a rate of about 16/sec. Anything faster than that, it has connection issues, and cpu load gets of the roof (total tasks & zombie processes amount start to accumulate. However memory usage seems low).
The error message that we see on the private server after a few seconds of high frequency connections is
ssh_exchange_identification: Connection closed by remote host
running the same test script on the external server, we don't see an issue with that rate (cpu load is about 7%, no zombies).
Assuming it is a ssh issue, so we added these in the ssh config file and restarted the ssh service but it didn't help:
MaxStartups 20:50:100 MaxSessions 100
As suggested, I tried to use ControlMaster. Here is what I did:
Added this to the ~/.ssh/config in the private server:
Host <external_host_ip> ControlMaster auto ControlPath ~/.ssh/cm_socket/%r@%h:%p ControlPersist 600
Created the first connection:
ssh -M -S /home/user/.ssh/cm_socket/user@host_ip:22 -i ~/key.pem user@host_ip
I checked, and something happened in the 'cm_socket' folder:
$ ll ~/.ssh/cm_socket/ srw------- 1 user user 0 Oct 18 18:10 user@host_ip:22=
but when I tried to do a subsequent ssh without the key, it didn't work:
$ ssh -M -S /home/user/.ssh/cm_socket/user@host_ip:22 user@host_ip Permission denied (publickey).
Is there a way to increase the throughput that can be handled by this setup?