I currently have an Ubuntu Server 16.04 LTS and I want to be able to receive email notifications with information about the resource usage on my server. CPU, memory, swap, you name it.
I understand that I can use Glances for local monitoring but I won't be on the console looking at the information all the time. I just need email notifications that inform me of resources that hit above normal/critical values and get notified.
Does anyone know of a method that allows me to do this?
I'll give you a small list of tools - while I would personally recommend Nagios simply as I am used to it. It comes with a big user base, good collection of plug-ins etc - but I've heard good things about Icinga as well.
Be aware that setting up Nagios or Icinga will take some time until you have a perfectly working setup which covers / monitors all your existing services. The start is most likely not that easy, but once you got it working it's a lovely setup which will notify for all your defined services if states do change so something which is reportable/defined as critical
First of all - a quick overview:
Other interesting software
Wikipedia writes
Nagios /ˈnɑːɡiːoʊs/, now known as Nagios Core, is a free and open source computer-software application that monitors systems, networks and infrastructure. Nagios offers monitoring and alerting services for servers, switches, applications and services. It alerts users when things go wrong and alerts them a second time when the problem has been resolved.
Example image - showing 1 of several views you have:
Here is a good AskUbuntu post about How to Install Nagios. Nagios Exchange offers a huge collection of plugins, addons, documentation, extensions, and more.
Wikipedia writes
Munin is a free and open-source computer system monitoring, network monitoring and infrastructure monitoring software application. It offers monitoring and alerting services for servers, switches, applications, services, etc. It alerts the users when things go wrong and alerts them a second time when the problem has been resolved
Example Image
Wikipedia writes
Icinga is an open source computer system and network monitoring application. It was originally created as a fork of the Nagios system monitoring application in 2009.
Icinga is attempting to get past perceived short-comings in Nagios' development process, as well as adding new features such as a modern Web 2.0 style user interface, additional database connectors (for MySQL, Oracle, and PostgreSQL), and a REST API that lets administrators integrate numerous extensions without complicated modification of the Icinga core.
Example Image showing 1 of several views:
Wikipedia writes:
Spiceworks is a professional network for the information technology (IT) industry that is headquartered in Austin, Texas. The company was founded in January 2006 by Scott Abel, Jay Hallberg, Greg Kattawar, and Francis Sullivan to build IT management software.
Spiceworks is an online community where users can collaborate and seek advice from one another and also engage in a marketplace to purchase IT-related services and products. The network is estimated to be used by more than six million IT professionals and 3,000 technology vendors.
Wikipedia writes
Monit is a free, open source process supervision tool for Unix and Linux. With Monit, system status can be viewed directly from the command line, or via the native HTTP(S) web server. Monit rose to popularity with Ruby on Rails and the Mongrel web server,[citation needed] because a tool was needed that could manage the many identical Mongrel processes that needed to be run to support a scalable Ruby on Rails site, and Monit was fairly uniquely suited for the needs of the Ruby on Rails community. Many popular Rails sites have used Monit, including Twitter and scribd.
Wikipedia writes:
Cacti is an open-source, web-based network monitoring and graphing tool designed as a front-end application for the open-source, industry-standard data logging tool RRDtool. Cacti allows a user to poll services at predetermined intervals and graph the resulting data. It is generally used to graph time-series data of metrics such as CPU load and network bandwidth utilization. A common usage is to monitor network traffic by polling a network switch or router interface via Simple Network Management Protocol (SNMP).
The front end can handle multiple users, each with their own graph sets, so it is sometimes used by web hosting providers (especially dedicated server, virtual private server, and collocation providers) to display bandwidth statistics for their customers. It can be used to configure the data collection itself, allowing certain setups to be monitored without any manual configuration of RRDtool. Cacti can be extended to monitor any source via shell scripts and executables.
Wikpedia writes:
Zabbix is enterprise open source monitoring software for networks and applications, created by Alexei Vladishev. It is designed to monitor and track the status of various network services, servers, and other network hardware.
Zabbix uses MySQL, PostgreSQL, SQLite, Oracle or IBM DB2 to store data. Its backend is written in C and the web frontend is written in PHP. Zabbix offers several monitoring options
You can use several solution. One of them is monit.
From command line, just run the following command:
sudo apt-get install monit
Monit should now be installed and accessible through one of the following URLs:
http://localhost:2812
http://IPADDRESS:2812 (local network IP)
http://domain.com:2812 (if you have domain name pointing to your server)
Monit Configuration
Before you can start using Monit for automatic server monitoring, you have to do some basic configuration. First backup the existing default Monit configuration using the following command:
sudo mv /etc/monit/monitrc /etc/monit/monitrc.bak
Next, create a new monitrc file using the command
sudo /etc/monit/monitrc
and copy the following contents to it.
# How often in seconds should monit check your services.
set daemon 120
set logfile /var/log/monit.log
set idfile /var/lib/monit/id
set statefile /var/lib/monit/state
# Configure your SMTP out server.
set mailserver smtp-server.columbus.rr.com port 25,localhost
set eventqueue
basedir /var/lib/monit/events # set the base directory where events will be stored
# optionally limit the queue size
slots 100
# Use one of the following 2 lines. The second line alerts on every little change and can be annoying.
set alert [email protected] but not on { instance, pid, ppid } #does not send alert on pid changes
#set alert [email protected]
set httpd port 2812 and
#Change username and password
allow Username:Password
# To enable SSL for WebUI uncomment the next 2 lines
#ssl enable
#pemfile /path/to/unified/certificate.pem
# To restrict access to localhost only uncomment the following line
#allow localhost
include /etc/monit/conf.d/*
Lines that begin with a # are comments to help you customize the configuration. Make sure you have at least the admin email, SMTP server, and SMTP port configured correctly. You should also consider changing Monit WebUI username and password. If you have an SSL certificate you can enable that as well for HTTPS access
For now use the following commands to ensure Monit is working well.
To test Monit configuration for syntax errors:
sudo monit -t
To start Monit:
sudo monit
To check Monit Status:
sudo monit status
System Load Monitoring with Monit
It is required that you have a working Monit instance with a proper /etc/monit/monitrc file. Monit configurations for various services are loaded from /etc/monit/conf.d folder. To monitor server load with Monit, create a Monit configuration file using the following command:
sudo /etc/monit/conf.d/systemload
Copy the following contents to it, save, and exit
# domain.com could be IP, hostname, or localhost
check System domain.com
if loadavg (1min) > 4 then alert
if loadavg (5min) > 2 then alert
if memory usage > 75% then alert
if swap usage > 25% then alert
if cpu usage (user) > 80% then alert
if cpu usage (system) > 30% then alert
if cpu usage (wait) > 20% then alert
This code will make Monit send you an email alert when one of the above conditions (eg. average load is >4 for at least 1 min or when more than 75% RAM is full) are met. You can customize the above rules as you please. Below is an example email alert sent by Monit along with a description of what condition caused the alert.
Test and Reload Monit
Once you make any changes you have to test Monit configuration:
sudo monit -t
You should see the following message: Control File Syntax OK.
Then, check to see if Monit is already running using the following command:
sudo /etc/init.d/monit status
If Monit is running, reload Monit configurations using the following command:
sudo /etc/init.d/monit reload
Now, fire up your web browser and visit one of the following URLs depending on how your Monit is configured (be sure to use the correct port number):
http://localhost:2812
http://IPADDRESS:2812 (local network IP)
http://domain.com:2812 (if you have domain name pointing to your server)
You should see the system status, load, CPU load, Memory Load, and Swap load
Storage Monitoring with Monit
Next, it is required that you have a working Monit instance with a proper /etc/monit/monitrc file. Monit configurations for various services are loaded from /etc/monit/conf.d folder. For drive space monitoring with Monit, create aMonit` configuration file using the following command:
sudo /etc/monit/conf.d/storagespace
Copy the following contents to it, save, and exit
# add each drive you want to monitor below
check filesystem Ubuntu with path /dev/sda1
if space usage > 90% then alert
check filesystem Home with path /dev/sda3
if space usage > 90% then alert
check filesystem Media with path /dev/sdb1
if space usage > 90% then alert
The code above code will monitor hard drive space in 3 partitions in 2 drives. If any of them is more than 90% full you will got alert.
Also you can monitoring a specific services.
Monitor Apache Server with Monit
Setting up Apache server monitoring with Monit is with Monit’s pre-made configuration templates. All you have to do is copy the existing template from /etc/monit/monitrc.d to /etc/monit/conf.d folder.
sudo cp /etc/monit/monitrc.d/apache2 /etc/monit/conf.d/
Instead of copying, you may also create a symbolic link. Apache web server process creates apache2.pid. The above code monitor apache2.pid file and if does not exist, Monit will try to restart Apache. A restart will trigger an email alert. If restart fails multiple times then Monit stops monitoring Apache server.
...
You can also have a look at SeaLion You can set customized alerts for CPU usage, memory etc. In addition, you can also add your own metrics and configure notifications for them as well. Also, you can schedule the commands to run at specific intervals and see their outputs in a timeline display. You can find more about it here.
The above tools are all great in generating e-mails. I prefer to get e-mails for my daily monitoring since I don't want to sit and watch a dashboard all day. The problem is you get flooded by e-mails and you are only interested in when something goes wrong. Also how do you know if an e-mail for some reason was never delivered and you think everything is working fine.
I searched a long time and came across 2 products that can read your report e-mails and warn you when you need to take action.
In my school, our teacher gave us a training of Pandora FMS, a monitoring solution that covers server monitoring, applications and even networks!
I only knew Zabbix and Nagios by the time, I've had never heard of this solution before, but now I'm basing my project in it because of the extra features it provides. The server monitoring part is really mature and in my opinion, it's the more simple to configure and to get performance stats ASAP: http://pandorafms.com/monitoring-solutions/server-monitoring/
Although it doesn't exactly do what you want out of the box, I find logwatch to be an extremely useful, yet simple tool.
It gives you a daily summary of all the events that have happened in your /var/log directory as well as some other important system information such as current disk space. Anything that is missing can be added quite simply (for example, I have added a script that checks certain daemons are still running).
From what I understand your need is very basic. You would like to text search email content for specific keywords and then get notified when this happends. So only when for example the email contains the word fail you want to be notified and not when it contains success. I also use www.smtpviewer.com as mentioned by John Grobbelaar to get notified . The advantage of using an online tool is you can always be notified when expected email does not arrive. From all the mail monitor tools I have played with this one is the simplest and only does this one thing.
