Datasource programs

Introduction

In some circumstances it is not possible or desirable to use the default method of transporting the data from the agent through an unencrypted TCP connection on port 6556. Some of those are:

• Monitoring via the internet (should be encrypted)
• Monitoring hosts in a DMZ (only SSH access is possible)
• Monitoring of operating systems that do not provide an easy method for binding a script to a TCP port

Check_mk allows you to specify a program to call instead of connecting port 6556. That program must output the data of the agent on stdout.

Example: monitoring Linux/UNIX hosts via SSH

The ssh client can be such a program. All you have to do is to install the check_mk_agent onto your target host into /usr/bin. Do not configure inetd or xinetd.

From the Nagios host you can call the agent easily with:

nagios@nagios:> ssh targethost check_mk_agent
<<<check_mk>>>
Version: 1.0.31
<<<df>>>
/dev/mapper/system-root ext3  28834812   9655560  17714524      36% /
/dev/md1      ext3      964408     46768    868648       6% /boot
/dev/mapper/system-home ext3  96118540  33698672  57537232      37% /home

Of course your Nagios user needs to be able to login without password. A howto for setting up ssh without password can be found here (German) or here (English).

Configuration in check_mk

Now let's tell check_mk to use ssh: Configuration of datasource programs is done in main.mk. The variable is named datasource_programs. The configuration is host-specific and its syntax is equivalent to host_groups.

You specify a list of entries. Each entry is a tuple of:

• a command line to call and a list of hosts that use that command.
• an optional list of host tags
• the list of hosts affected by that entry

The command may contain the special tag <IP> which will be replaced by the IP address of the host:

datasource_programs = [
 ( "ssh -l root <IP> check_mk_agent", [ 'web01', 'web02' ] ),
]

If you need different SSH keys for different hosts, you can solve this by using one declaration for each key:

datasource_programs = [
 ( "ssh -i /somedir/dmz.key -l root <IP> check_mk_agent", [ 'web01', 'web02' ] ),
 ( "ssh -i /somedir/vms.key -l root <IP> check_mk_agent", [ 'vmsx1' ] ),
]

As of version 1.1.3, it is also possible to use the tag <HOST>, which will be subsituted with the host name of the host being checked. This allows checks of hosts with no or several IP addresses using self made helper scripts.

Using host tags

If you have more than two or three hosts to deal with, host tags provide an easy method for selecting which hosts to contact via ssh. Just append the a tag - say ssh - to each host that should be contacted via ssh:

all_hosts = [
 'xyhost1',
 'xyhost2',
 'xyhost3',
 'web01|ssh',
 'web02|ssh'
]

Then you can use that tag in datasource_programs:

datasource_programs = [
 ( "ssh -l root <IP> check_mk_agent", ['ssh'], ALL_HOSTS ),
]

Please note, that an entry using host tags has not two but three components. The list of host tags comes before the host list. The special tag ALL_HOSTS represents all hosts - regardless of the current content of the variable all_hosts.

Restricting SSH and making it safe

Why do we need to make it safe?

The check_mk_agent needs to be run as root. This is not nice, but neccessary, since some of the information it gathers can only be read with root priviledges (for example some logfiles and network interface parameters).

Having an ssh key lying around on the Nagios server that allows full root login on some of your machines without password is not at all desirable. But you do not have to: simply restrict logins with the key that Nagios is using to just run /usr/bin/check_mk_agent - regardless of how ssh is called. This is how to do it (with openssh):

Setting up ssh with command restriction

Create a new pair of keys especially for monitoring with check_mk

root@linux# cd /etc/check_mk
root@linux# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): check_mk.key
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in check_mk.key.
Your public key has been saved in check_mk.key.pub.
The key fingerprint is:
d8:db:b0:8c:db:df:33:a0:ba:e9:b1:30:4a:c3:d0:27 root@Nagios
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
| .     o         |
|. E . . S        |
| o o   o =.      |
|  + o o +...     |
| . o o *.  .o    |
|  .  .Bo... .o   |
+-----------------+

Make the private key readable for Nagios:

root@linux# chown nagios check_mk.key
root@linux# chmod 400 check_mk.key

Install that key on your target machines but restrict it to executing /usr/bin/check_mk_agent. This is done by prepending the key line with command="/usr/bin/check_mk_agent" (all in one line!):

command="/usr/bin/check_mk_agent" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQE
A06ssYDmr5eWxbp+piT4vyPBO3qYsk9ZsagAgZ4ygmGQzSZr9yxaZSkUFFXJvkwZuRVPI
U5UITBi3Whvh2y5Ajm91+M5qKwXeakrJ49GlsyuLfzYr7onttfP0cleLQuLLrf5Pxly1g
yaDMN2esPDkngl/XplvaWJQDmQAAQmgMzYig+J+xGHESU9IomUHSZ/oju5xiFVVA+gLMg
6BWPnYDA2zwGLuYZmeRdw+qJN4Mg6jiD5XQPqBVSI+zlcyFbaAz2EVlLEIRmx1nPOtVb/
SdxNr7VEYkted6WDXmm6KYMvnzWtJCe8mdIgNhCKd4pFKniJNUUhtK7GFiaF3/agJyQ==
 root@Nagios

Testing it

After that try to login via ssh:

root@linux# ssh -i check_mk.key targethost

You should not get a shell but the output of the agent now. If an attacker gets hold of the privat key all he can do is to read the output from the agent.

Integration into check_mk

Integration into check_mk is not special here. Do not forget to specify the key with ssh's -i option. You can omit the command. It's executed automatically:

datasource_programs = [
 ( "ssh -l root -i /etc/check_mk/check_mk.key <IP>", [ 'web01', 'web02' ] ),
]