How to monitor network cards on Icinga2? (part-1)

Yesterday I've been on a task of monitoring network cards of all our servers and infrastructure to check the bandwidth in/out and send alarms based on some criteria. In Icigna2 we have a library from nagios called check_nwc_health. Download the script from https:// labs.consol.de/nagios/check_nwc_health/index.html.

Move the script to /usr/lib/nagios/plugins on a server that you have installed Icinga2. If you run it all alone you will get some helps that you could be useful.

Some important usages of the script:

- list interfaces of a specific server (we assume snmp has been installed on the destination server):

./check_nwc_health --mode list-interfaces --hostname YOUR_TARGET_SERVER_IP  --community YOUR_COMMUNITY_STRING

The output would be something like below (it can be different in your case):

000001 lo
 000002 Device 1af4:0001 2
 000003 Device 1af4:0001 3
 000004 docker0
 OK - have fun

The interface name is given in front of serial numbers which is lo, Device 1af4:0001 2 or docker0. These interface names are important and will be used in icinga2 to add network card to hosts.


Another mode for the script is interface-usage that shows in/out bandwidth. The output can be something like follow:

OK - interface Device 1af4:0001 2 (alias eth0) usage is in:0.00% (7058.67bit/s) out:0.00% (5603.67bit/s) | 'Device 1af4:0001 2_usage_in'=0%;80;90;0;100 'Device 1af4:0001 2_usage_out'=0%;80;90;0;100 'Device 1af4:0001 2_traffic_in'=7058.67;0;0;0;0 'Device 1af4:0001         2_traffic_out'=5603.67;0;0;0;0

OK, the important part is over and we can list all server network interfaces plus the usage of a specific network interface. In the next part we will explain the Icinga2 part to add the command and the service to icinga2.


#icinga2 #icinga #nagios #check_nwc_health #network #monitor

How to monitor network cards on Icinga2? (part-2)

Ok for now we have added the plugin to nagios folder and ran some tests on target server's network interfaces. We need to add a command to Icinga2 to use it in service section of Icinga2. To create a new command create a new file in /etc/icinga2/conf.d/commands/check_nwc_command.conf and with the following content:

object CheckCommand "YOUR_COMMAND_NAME" {
     import "plugin-check-command"
     command = [ PluginDir + "/check_nwc_health", "--mode", "interface-usage" ]
     arguments = {
         "-H" = "$address$"
         "-C" = "$community$"
         "--name" = "$int$"
     }
 }

In brief it creates a new command called YOUR_COMMAND_NAME that calls the script check_nwc_health with interface-usage argument to get the bandwidth data.


Now we need to use this command in a service. We have to create a new service which will be used in our hosts configuration sections /etc/icinga2/conf.d/services/if_traffic.conf:

apply Service for (display_name => config in host.vars.int) {
     import "generic-service"
     check_command = "YOUR_COMMAND_NAME"
     vars += config
     assign where host.vars.int
 }

Again in brief the service will be applied on hosts that have a variable section of int in their configuration that we will see a little bit later. YOUR_COMMAND_NAME is the name that we have given in the first part when creating the command.


The final part is to add this service to your desired host. Go to /etc/icinga2/conf.d/hosts and open the file which relates to your host. Host files content start with:

object Host "host-54 (Infra)" {

Add the service like below into your host:

vars.int["YOUR DISPLAY NAME"] = {
     int = "Device 1af4:0001 2"
     community = "YOUR SERVER COMMUNITY STRING"
 }

int is the part that we give the interface name, this should be given from the output of list-interfaces in part-1.

You can go even further like me :) and add these data into Grafana dashboard to have a better understanding of what is happening around you.

#icinga2 #icinga #service #host #command #nagios #interface #network

In case your upgrade failed, try to start icinga service and in the meantime check the logs in another console:

service icinga2 start

Check logs to see what is the error:

tail -f /var/log/icinga2/icinga2.log

If it point to a specific config file, change the config as reported or in case you don't need it remove it. It happened for me on another Icinga2 server and I removed the old config and tried to apply the new schemas. It gave another error on mysql grants, I gave all permissions to the icinga2 user:

mysql -u root -p
 GRANT ALL PRIVILEGES ON icinga2_database_name.* TO 'icinga2_username'@'localhost'
 flush privileges;

Now all things should be up and working now.

#icinga2 #monitoring #upgrade #schema_update #icinga