0

I occasionally use this line in my terminal to see which user-agent is using my server more.

cat /var/log/apache2/access.log | awk -F\" '{print $6}' | sort | uniq -c | sort -n

Scans the access.log and show in ascending order the user-agents that it has found several times.

The result is something like this:

  10283 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1
  23247 Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
  40063 MauiBot (crawler.feedback+dc@gmail.com)
 143724 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
 192741 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

I have no idea how it works, so long ago I found it somewhere or someone gave me the date, I do not remember.

Anyway, is possible do the same thing but ordering for IP?

  • Where did you find the command-line you are using? Could you post a link? BTW, there is a Useless Use of Cat. – Melebius Jul 26 '18 at 8:03
  • I don't remember... use it from a lot of time – alebal Jul 27 '18 at 0:26
0

I think IP is the on the first column in the log, try

awk '{print $1}' /var/log/apache2/access.log | sort | uniq -c | sort -n
  • Umh.. no, a lot of time the same ip... – alebal Jul 27 '18 at 0:29
  • It should give you a list of ip with a count in front of it. make sure the access log has not just rotated, if so you will need to parse older log files. – Bernard Wei Jul 27 '18 at 0:34
0

Judging from the fact that you're using awk -F \" { print $6} there, this appears like combined log format, which should have the IP address in field $1. From there, we can just do uniq in awk, then combine that with sorting indices in associative arrays.

With a mock access.log file like this:

127.0.0.1 - - [05/Feb/2012:17:11:55 +0000] "GET / HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.5 Safari/535.19"
192.168.0.2 - - [05/Feb/2012:17:11:55 +0000] "GET / HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.5 Safari/535.19"
127.0.0.1 - - [05/Feb/2012:17:11:55 +0000] "GET / HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.5 Safari/535.19"

the awk code should be as so:

$ awk '{!a[$1]++}; END{n=asorti(a,sorted);for(i=1;i<=n;i++) print sorted[i],":",a[sorted[i]]}' access.log
127.0.0.1 : 2
192.168.0.2 : 1

For numeric sorting by the number of entries for each ip address, we can just stick sort at the end:

$ awk '{!a[$1]++}; END{for(val in a) print val,":",a[val];}' access.log | sort -n --field-separator=":" -k 2,2

Of course, you should use /var/log/apache2/access.log as your actual input file.

  • This seems to work, but only shows IP and how many times it has entered, I would also like to know who is, maybe from the user-agent, is it possible? – alebal Jul 27 '18 at 0:34
  • @alebal Know who is ? As in making a querry via whois command ? You might want to ask a separate question maybe – Sergiy Kolodyazhnyy Jul 27 '18 at 0:40
  • I would like to use it for example to identify and block the bad bots, if there was also the user agent would be much more comfortable, is not possible? – alebal Jul 27 '18 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.