How to use syslog for else output ( show up with whois - query )?

Question

There is log-file /var/log/syslog with the output of ip-addresses like e.g. SRC=10.158.0.1.

I want to scan from this file every printed address in SRC= ... and then to make query with whois (SRC= ...).

This query should be monitored.

Mine attempt is wrong and looks like this in a bash-script:

#/bin/bash

while [ 1 ]
do
    grep ‘SRC=ip-address’ /var/log/syslog >> /home/$user/topsecret001/pitbull001.txt;
    whois ‘SRC=ip-address’ >> /home/$user/topsecret001/pitbull002.txt;
done

Can somebody help with a trick ? How should I define ip-address and how can I use command whois with this ip-address ?

The output of /var/log/syslog looks like this snippet here :

http://paste.ubuntu.com/5859332/

The output of /var/log/syslog looks like this - when there is a little alert (like today) :

http://paste.ubuntu.com/5862958/

Idea would be too - to perform a whois-query only if there is "invalid state" popping up in the line of the scrolling syslog.

---

Thank you for your contributions. I have learnt something by your codes. Often the solutions are looking easier than thought like here - cause I thought it would be more difficulty. I think with recent contribution of enzotib then this question is solved already now.

---

see new comment of today (22nd June 2016) referring to 16.04 :

because this bash-script was for times of ipv4 - does then /etc/sysctl.conf needs to be un-commented to enable ipv4 ? - then this script would run ? otherwise there is no whois-output any more like before. Have checked this with uncommenting line 28 and line 33 of /etc/sysctl.conf - then this bash-script in this thread would work, but deliver very few output because provider has enabled firewall (because of no-spy-act ?). This way by this addition of 22nd June 2016 this thread is actualized for 16.04

Answer 1

Yet another solution:

awk '{ 
      for (i = 1; i <= NF; i++) 
        if ($i ~ /^SRC=/) 
          print substr($i, 5) 
    }' /var/log/syslog |
  sort -u |
  while read ip; do
    printf ' === %s ===\n' "$ip"
    whois "$ip"
  done

If you only want to select lines of syslog containing the string INVALID STATE, then the above code can be modified as follows

awk '/INVALID STATE/ { 
      for (i = 1; i <= NF; i++) 
        if ($i ~ /^SRC=/) 
          print substr($i, 5) 
    }' /var/log/syslog |
  sort -u |
  while read ip; do
    printf ' === %s ===\n' "$ip"
    whois "$ip"
  done

Answer 2

You can use a sed command to extract all of the ip the IP addresses from the file, then use xargs to run whois for each match:

sed 's/^.*SRC=\([0-9.]*\).*$/\1/;t;d' < /var/log/syslog | xargs -n1 whois >> output.txt

The s/^.*SRC=\([0-9.]*\).*$/\1/ command replaces lines that contain SRC=x.x.x.x with just the IP address.

The 't;d' commands skips non-matching lines (thereby avoiding a separate grep command).

The xargs command invokes whois once for each address that sed outputs.

---

Alternatively, you can find and log the matching lines first, then extract the ip addresses separately:

grep -eSRC=[0-9.]* /var/log/syslog | tee grep-output.txt | sed 's/^.*SRC=\([0-9.]*\).*$/\1/' | xargs -n1 whois >> whois-output.txt`

Answer 3

Maybe the following code is a suitable starting point for you. It probably isn't the optimal solution, but it does its job.

It consists of a for loop over all lines of output of the command within $(). In each iteration of the loop, one line of output is stored in the variable IP. Then, in the loop, the whois command is called with $IP - the content of the variable IP - as argument.

The brackets $() enclose two grep commands - the first one is searching for IP addresses with SRC= written in front of them and the second one takes the output of the first one (via a pipe |) and just takes the IP address. The -o flag of grep causes it to only output the matched part of the lines instead of the full lines.

The regular expression is also not very elegant yet. It is searching for three groups, each consisting of one to three digits and a dot, followed again by one to three digits. To keep the script readable, I chose to use -E extended regular expressions. The "normal" grep command would require a backslash in front of every round and curly bracket...

for IP in \
 $(grep  -E "SRC=([0-9]{1,3}\.){3}[[0-9]{1,3}" -o  /var/log/syslog  | \
 grep -E "([0-9]{1,3}\.){3}[[0-9]{1,3}" -o);
    do whois $IP;
done

Answer 4

Your script should look like this:

#/bin/bash

grep 'SRC=' /var/log/syslog | awk 'BEGIN {FS="[ \t]+|\\|"} {print $13}' | uniq >> ~/topsecret001/pitbull001.txt

for ip in $(sed -e 's/SRC=//g' ~/topsecret001/pitbull001.txt)
do 
    whois $ip >> ~/topsecret001/pitbull002.txt
done

Answer 5

For to add option - in case of popping up "INVALID STATE" - would then work this ? :

awk '{

    for (i = 1; i <= NF; i++)

      if ($i ~ /^SRC=/)

         print substr($i, 5)

   }' /var/log/syslog | sort -u | while read ip; 

do

   printf ' INVALID STATE ' && printf ' === %s ===\n' "$ip"

   whois "$ip"

 done