5

How do I only show the unique IP addresses in /var/log/auth.log ? I have a server getting hit constantly, and while I have set up fail2ban, the requests keep coming. So I started writing to ISPs to get the accounts shutdown, which might help a little, but there are so many attacks.... Anyway,

I used this command to sort the hits from today: $grep -e 'ruser \| rhost' auth.log | grep -e 'Jun 9' | sort -k 14 | less its thousands of attacks... or brute force attempts.

Heres some relivent lines from auth.log:

...
Jun  9 10:08:35 nan sshd[23397]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53
Jun  9 10:08:51 nan sshd[23401]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53
Jun  9 10:09:07 nan sshd[23428]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53
Jun  9 10:01:59 nan sshd[23216]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:02:52 nan sshd[23238]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:03:08 nan sshd[23246]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:03:40 nan sshd[23256]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:04:14 nan sshd[23270]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:04:31 nan sshd[23276]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:05:23 nan sshd[23298]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:05:55 nan sshd[23315]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:06:49 nan sshd[23343]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:07:47 nan sshd[23375]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:08:02 nan sshd[23383]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:09:47 nan sshd[23440]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:10:02 nan sshd[23444]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=root
Jun  9 10:09:27 nan sshd[23436]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.109.161.53  user=sshd
Jun  9 05:27:00 nan sshd[20293]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.134.44.86
....

I would like to make a list of the unique IP addresses in auth.log.

I tried using gawk -F: '{ print $14 }' auth.log | uniq -c

but I guess I'm doing something wrong. I just want a sorted list of IP addresses from today. how do I do that?

1
  • 4
    I hate to be the bearer of bad news but in response to your statement of "I have a server getting hit constantly, and while I have set up fail2ban, the requests keep coming", this is The Current Standard Traffic on the Internet. If your system is Internet facing, you will be bombarded repeatedly by service scanners, brute forcers, etc. on your network, and though fail2ban works, you will NEVER cease to see these requests. The only way to protect yourself is proper server hardening to prevent password bruteforce.
    – Thomas Ward
    Jun 9, 2023 at 16:17

3 Answers 3

6

uniq only detects adjacent duplicates - you'd need to sort first

awk '{print $14}' auth.log | sort | uniq

or use the sort command's own -u option

awk '{print $14}' auth.log | sort -u

or (since you're already using awk)

awk '!seen[$14]++ {print $14}' auth.log

(The rhost= entry is the 14th whitespace delimited field - no need for -F:)

1
  • 1
    Or maybe awk -F'[= ]' '/ruser/ && /rhost/ { print $21 }' auth.log | sort -u to get IPs only with conditions.
    – Raffa
    Jun 9, 2023 at 16:19
3

With grep

I see you're struggling a bit with grep matching patterns as you appear to need to match some patterns before printing unique IPs ... So, ...:

I would use grep's option -o to only print the matched part and enable Perl style regular expressions with -P and use:

  • Jun to match Jun and then ...
  • \s+ to match one or more spaces and then ...
  • 9 to match 9 and then ...
  • .* to match anything until ...
  • ruser= is matched and then ...
  • .* to match anything until ...
  • rhost= is matched and then ...
  • exclude all the above from printing with Perl's \K and then ...
  • match an IP with \d+\.\d+\.\d+\.\d+ where \d+ matches one or more digits and \. matches a literal dot and then ...
  • pipe the output to sort -u to print only unique IPs like so:
grep -oP 'Jun\s+9.*ruser=.*rhost=\K\d+\.\d+\.\d+\.\d+' auth.log | sort -u

With gawk

Or since you appear to have GNU AWK installed, you can do it all(including sorting ... utilizing Predefined Array Scanning Orders) with gawk like so:

gawk 'BEGIN {
    FS = "[= ]*"
    PROCINFO["sorted_in"] = "@ind_num_asc"
}

/Jun\s+9/ && /ruser/ && /rhost/ {
    a[$18]++
}

END {
    for (i in a) {
        printf "%s\n", i
    }
}' auth.log
0
$ egrep -o '[1-2]?[0-9][0-9]?\.[1-2]?[0-9][0-9]?\.[1-2]?[0-9][0-9]?\.[1-2]?[0-9][0-9]?' /var/log/auth.log \
|sort -u

I made up the regex pattern, but any regex pattern that matches IP address (there are lots online) should be allowed.

Useful for searching any logs for IP addresses.

Raffa's answer is a bit cleaner, though the first digit in each segment of an IP can only a 1 or 2. (ignoring IPv6 addresses).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .