1

I have the current solution, fail2ban and forbid SSH as root in the sshd configs. Its not as effective as I need in certain situations. Most particularly in gateway servers, that have limited memory and disk space (because they are supposed to be lightweight)

Disabling login as root in the sshd configs, still permits the bots to connect, specify login as root, and try 3+ times. Fail2ban then blocks their IPs after 5 failures.

However, the incessant volume of bots then leaves 8 sshd threads in memory at any time, 3GB auth.log of failures (30% of my disk space), huge memory overheads for fail2ban to filter and process them all, and slow response when we attempt to login because there are 50,000+ ip blocks each connection must be filtered through first, and 20-48MB of memory used for loggin and security are on swap because of the system demands to handle the volume of requests.

The preferable solution is: "When an SSH connection attempts to login and user=root" then "sshd disconnect". Any attempt to specify the user root results in dropping the connection.

This would reduce the unnecessary processing to filter all the brute force attacks out. I cannot use keys-only access because it causes the login to by-pass the required 2 factor authentication.

  • SSH keys do not disable PAM from working. I use GAuth+SSH keys on my own server without an issue. Perhaps something in your PAM config is amiss there? – Kaz Wolfe May 8 '17 at 17:27
  • I could be missing a configuration for that. The steps I took: Install sshd, configure (key or pass), Install libpam-google-authenticator, configure (it worked as password), add my ssh keys, ssh in = no google authentication request. Delete ssh key, given google authentication request. – ppostma1 May 8 '17 at 17:35
  • 1
    Take a look over at this serverfault thread, you need to specify PAM, challenge-response, and keyboard authentication modes. – Kaz Wolfe May 8 '17 at 17:38
  • those configurations changes did work. Unfortunately, I have this in the header of the ssh-config: "When connecting to your server, HOSTAGENTNAME will login as the user 'username' using PasswordAuthentication on port 22." That is unfortunate, we have risks turning password authentication off – ppostma1 May 8 '17 at 17:51
  • It only worked until we rebooted the box. So the options are password + challenge response, publickey + challenge, but attempting to set (password | publickey) + challenge results in (password + challenge) | publickey, or results in publickey + password + challenge. THEN if F* updates find you have challenge and publickey set, they will automatically disable passwords on your behalf by commenting it out in the configs – ppostma1 May 11 '17 at 16:13
1

personally, I prefer to change ssh listening port, it can avoid this problem. And it is really easy to do.

  • My personal preference as well, but I failed to clarify: this is a monitored, certified, public service point. Our webhosts must be able to admin and manage to keep our certifications. Its getting a bit ridiculous, that the requirements to meet the certification leave us open to attacks, that the certification is supposed to prove we are resistant to – ppostma1 May 10 '17 at 16:04
  • oh, sorry, but I never used fail2ban so I can't tell you another way to prevent ssh flooding. – iTzWam May 10 '17 at 16:07
  • 1 year later, I do both, fail2ban + port changed, and no more ssh bot trying to connect – iTzWam Jan 2 '18 at 17:50
0

It turns out that fail2ban can be modified to include this as an unwanted event. Change/add a ssh config file in filter.d to include a filter for access as root: https://serverfault.com/questions/340565/how-to-block-all-root-login-attempts-using-denyhosts-and-or-fail2ban

Then fail to ban will more-quickly respond to those logins, but not as fast as dropping them on the spot (which it turns out would require changing code and recompiling sshd)

Lowering the bantime to a few hours prevent a huge list of blocked ips the firewall would have to sort through, reducing overhead and connection time. The failed password, and login-as-root bantime can be configured separately. This way if a user has CAPS on, its only a few minutes, but brute force bots get blocked for 2 hours. (noticed the default bantime is 10 minutes, and most bots simply wait 10:01 minutes and resume their attempts)

Reducing the log-levels cuts back on all the log files and their memory overhead. Reducing the fail2ban findtime, reduces the amount of logfiles fail2ban holds in memory!

Having the correct settings is important, because all 3 of these systems create a memory usage love triangle! Not blocking the bots soon enough, results in huge log files which means more memory usage by rsyslog. Fail2ban holds these logs in memory to scan (# of attacks/second * by findtime). And the sshd threads backup from all the concurrent attempts the bots are permitted to make. Keep the bantime too short and the bots come right back creating more sshd threads. Have bantime set too long and the firewall has to run through a long list of permissions held in memory.

bantime: 5-10 minutes for bad passwords
bantime: 2-6 hours for attempts as root (not days, certainly not 99999999)
log-files: Warn (not info!)
findtime: 5-15 minutes on SSH (log memory usage reduced by quick bans)
maxretry: 2-3
Also config the max attempts permitted in sshd to be 2 because it will dump them to the watched log file much faster

The upcoming change in brute force attacks (which is why it was harder on us) is to run them concurrently. If you are going to block them after some attempts are made, then start 20x ssh connections and try 20 brute forces at once! All the configs above are only effective against a single ip, single connection attack. They are less effective if multiple ssh connections are run at the same time.

This can be blocked at the firewall preventing more than N connections from a given IP at one time: https://serverfault.com/questions/275669/ssh-sshd-how-do-i-set-max-login-attempts

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.