Was my SSH server compromised? If so, how, and what steps should I take?

Question

I recently enabled ssh access to my machine to work on a group project with my classmates. I read the guide for setting up ssh on ubuntu and tried to follow good security practices. I disabled password authentication, so the only way to get in was with RSA keys, and I only had 2 keys listed in the authorized_keys file: my own (used it while testing to see if ssh was working properly) and a friends.

This evening I was curious to see if my friend was ssh'd into my system while I was using it, so I googled for a command that would tell me if anyone was ssh'd in, and if so, who. The result I got was:

sudo netstat -tnpa | grep ESTABLISHED.*sshd

I tried it, and the output was:

tcp        0      0 192.168.1.86:22         59.47.0.150:44728       ESTABLISHED 7416/sshd: [accepte

That didn't look right. I contacted my friend and he assured me he wasn't logged in. I tried the command again, and saw:

tcp        0      0 192.168.1.86:22         59.47.0.150:44728       ESTABLISHED 7416/sshd: root [pr

At this point I was a little freaked out by the word "root" and messaged a friend who knew more about this stuff than I did. He told me to try:

 ps aux | grep ssh

which outputted:

root      3702  0.0  0.0  61364  2872 ?        Ss   Apr12   0:00 /usr/sbin/sshd -D
root      7473  0.0  0.0 112692  3920 ?        Ss   20:46   0:00 sshd: root [priv]   
sshd      7474  0.0  0.0  62784  1516 ?        S    20:46   0:00 sshd: root [net]    
sid       7476  0.0  0.0  22476   936 pts/1    S+   20:46   0:00 grep --color=auto ssh

Now I was thoroughly freaked out, so even though I wanted to wait a bit and see if I could find out about who was logged in, I decided to just sudo stop ssh.

After some more googling, I found out that 59.47.0.150 is an IP in/near Shenyang, China, and seems to be specifically known for malicious attacks.

So my questions to you guys are:

1. Can we say with certainty that an IP from china somehow SSH'd into my machine? Even though it only accepted RSA key authorization?

2. Can we say with certainty that he/she got root access too? In my ssh-config, I had the default PermitRootLogin without-password. I originally thought this meant that root login wasn't permitted (I know the two sound contradictory, but I had googled it and that was the result I got)

3. If so, how?

4. Is there any way I can see what damage has been done so far?

5. How can I safeguard against this in the future? I still need to have ssh running eventually to complete my group project.

Thanks for any help you can offer!

EDIT: As per saiarcot895's and Steven's suggestion, I checked auth.log, which had the following lines repeated a lot:

    Apr 13 20:43:50 PrometheusU sshd[7392]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=59.47.0.150  user=root
Apr 13 20:43:55 PrometheusU sshd[7394]: reverse mapping checking getaddrinfo for 150.0.47.59.broad.bx.ln.dynamic.163data.com.cn [59.47.0.150] failed - POSSIBLE BREAK-IN ATTEMPT!
Apr 13 20:43:55 PrometheusU sshd[7394]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=59.47.0.150  user=root
Apr 13 20:43:58 PrometheusU sshd[7394]: Failed password for root from 59.47.0.150 port 54813 ssh2
Apr 13 20:44:03 PrometheusU sshd[7394]: message repeated 2 times: [ Failed password for root from 59.47.0.150 port 54813 ssh2]
Apr 13 20:44:04 PrometheusU sshd[7394]: Received disconnect from 59.47.0.150: 11:  [preauth]

Does this mean that the attacker has gotten into my system and failed to login to root, or that he/she is trying to directly access root, has failed, and hasn't accessed anything at all?

Answer 1

Can we say with certainty that an IP from china somehow SSH'd into my machine? Even though it only accepted RSA key authorization?

It is unlikely they were authenticated, unless they managed to obtain a valid key. Try connecting from a host you control without a key and check the running processes. They should be similar to what you saw. The fact the [net] process is running as sshd indicates that they have not succeeded in logging in.

Can we say with certainty that he/she got root too? In my ssh-config, I had the default PermitRootLogin without-password. I originally thought this meant that root login wasn't permitted (I know the two sound contradictory, but I had googled it and that was the result I got)

The option you used permits logins to root, but disables password authentication for root. It is good practice not to allow any root logins via SSH unless you really need them. If you do restrict access as tightly as you can. Unless you have enabled a key, it is unlikely you can login to root using SSH. The without-password disables password based logins, but does not disable the password authentication method.

If so, how?

You can check the auth log to see if any authentications succeeded. However, if they did succeed they may have overwritten the evidence.

Is there any way I can see what damage has been done so far?

Running a rootkit checker and verifying file signatures offline should indicate if any files were compromised. Taking file signatures and storing them offline while you are secure will give you a baseline.

How can I safeguard against this in the future? I still need to have ssh running eventually to complete my group project.

sshd supports use of tcpwrappers to restrict access. Build a rule set that only allows access from address ranges that you expect traffic from.

Answer 2

Lots of ssh brute force attacks taking place. It is possible you have an obsolete ssh package and they got in that way.

Check for ssh failures in the logs, he maybe simply doing a brute force against you.

go to /tmp and post its output from ls -al, if there is a root kit it might show up there.

You can set allow users in ssh and fail2ban is useful as well.

Set /tmp and /home as non-executable in fstab if possible will help a lot.

Answer 3

If it is only a brute force attack filling up logs, simply change the port.

Most scripts are stupid and there are enough servers which listen to port 22. I always set mine to e.g. 30200 or 225. This can be done with ease in the

/etc/ssh/sshd.conf

Just change the port number and restart the daemon.

The system then can be accessed with an additional option to ssh

ssh -p224 user@example.com

Ma logs always were filling up, I changed the port and the noise was gone.