How we can monitor the network traffic of logs transferred through the rsyslog

Question

I'm consolidating the client logs to a centralized server in Ubuntu using the rsyslog deamon. While consolidating the logs, how can i see the amount of network traffic or file size transferred from the client to server?

Is there any way i can do filtering from the client machine before transferring the logs to server?

Answer 1

Forwarding only selected messages is easy. Put a file with the extension *.conf in /etc/rsyslog.d. Name it e.g. /etc/rsyslog.d/40-forwarder.conf to have it read before the defaults (in 50-default.conf) or /etc/rsyslog.d/60-forwarder.conf to have it read after the defaults:

if ( ($programname == "my-binary") or ($syslogfacility-text == "local1") ) then {
    action(type="omfwd" target="my.syslog.server" port="514")
}

This will forward all mesages with the given properties to host my.syslog.server, UDP port 514. If you put stop after the action, then no further processing is done for that message.

This means: if you name the file 40-....conf and add stop:

if ( ($programname == "my-binary") or ($syslogfacility-text == "local1") ) then {
    action(type="omfwd" target="my.syslog.server" port="514")
    stop
}

then the default actions for these messages (like writing them to /var/log/messages) will not take place.

As for you other question about network traffic: That is actually another question and should be posted separately.

---

Update

The condition

if ( ($programname == "my-binary") or 
     ($syslogfacility-text == "local1") ) then {…}

is just an example for a filter. Probably you have some other criteria to select the messages that should be forwarded to the centralized logging host.

In this case only those logmessages that were written to the logfacility local1 or were issued by some program called my-binary would be forwarded. All others whould not. You can test the filter with logger (the -t switch mimicks the program name):

logger -t my-binary    -plocal4.info "This is forwarded ($RANDOM)"
logger -t other-binary -plocal1.info "This is forwarded ($RANDOM)"
logger -t other-binary -plocal4.info "This is NOT forwarded ($RANDOM)"

On the remote host, the first two messages should appear, but not the last.

You didn't say what kind of filter should be applied, so I chose an arbitrary criteria.

See Filter Conditions and RainerScript for further information on what is possible with rsyslog.