Network Monitoring Appliance - Page 2

Want to support HowtoForge? Become a subscriber!
 
Submitted by gbi (Contact Author) (Forums) on Mon, 2009-09-28 17:17. ::

2. Lighttpd

As Nagios, MRTG, Smokeping and Weathermap4rrd use Lighttpd as common component to show their measurement results to the outside, in this step Lighttpd will be basically configured. All applications will be called via CGI scripts, so at least the standardmodule mod_cgi of lighty is necessary.

Therefore add the following to the section server.modules of /etc/lighttpd/lighttpd.conf:

            "mod_auth", 
            "mod_setenv", 
            "mod_cgi" 

As Nagios also needs the modules mod_auth and mod_setenv, they should also be added.

For the mod_cgi should be checked, if there is a symlink inside of /etc/lighttpd/conf-enabled pointing to /etc/lighttpd/conf-available/10-cgi.conf:

ls /etc/lighttpd/conf-enabled

total 0 
lrwxrwxrwx 1 root root 40 2009-08-13 11:06 10-cgi.conf -> /etc/lighttpd/conf-available/10-cgi.conf

After that have a look into /etc/lighttpd/conf-available/10-cgi.conf and check, that an alias for cgi-bin exists, which points to the location in the filesystem, where the CGI-scripts really are:

alias.url       += ( "/cgi-bin/" => "/usr/lib/cgi-bin/" )

The CGI-scripts are located outside the documentroot of Lighttpd.

Logging of Lighttpd is done under /var/log/lighttpd/.

 

3. Smokeping

Smokeping realizes a simple method to estimate latency times, paketloss and the like in a networked environment. In the simplest case a destination is pinged every 5 minutes with 20 ICMP_ECHO_REQUEST's, and the timely diversion (meanvalue and standarddeviation) of the replytimes of the ICMP_ECHO_REPLY's are visualized. History over time is realized with the help of RRDTool. Therefore Smokeping runs as service/daemon, and is automatically started on system boot. Configuration of destinations and probes is done in the config file /etc/smokeping/config. The visualized measurements are viewed in the webbrowser, and delivered via Lighttpd to the client running the browser.

For a complete documentation of all possibilities of smokeping please have a look at the site of its maintainer, where you can also find some examples: http://oss.oetiker.ch/smokeping/doc/index.en.html.

Start/Stop/Restart of the service:

/etc/init.d/smokeping start|stop|restart

Check of the syntax of the configfile:

smokeping -check

 

4. MRTG

Basically MRTG is used to visualize the workload of routers in transmission networks. The visualization is done in a way that both short periods (several hours) of time are viewed in more detail, and longer periods (months and years) are visualized with less details (for instance to see trends for capacity planning). This is realized by reading counters via SNMP from the routers in (default) intervals of 5 minutes, stored and visualized in form of .png files. These .png files are typically delivered to the outside world via a webserver and viewed in a webbrowser. The values of the counters are stored either inside of ASCII Files (default), or better inside of Round-Robin Databases. The possibilities of MRTG regarding such measurements and visualizations are nearly infinite, as not only SNMP could be used as source of data, but also self written scripts or programs. Some examples can be seen on the site of its maintainer: http://oss.oetiker.ch/mrtg/ where also the complete documentation could be found under http://oss.oetiker.ch/mrtg/doc/index.en.html.

Configuration is done in the ASCII File /etc/mrtg.cfg. Differing from the default "LogFormat: rrdtool" is used (see http://oss.oetiker.ch/mrtg/doc/mrtg-rrd.en.html). This has 2 objectives, performance and reusability.

In a defaultinstallation all destinations are polled in 5 minute intervals, measured values are stored and .png files will be created. Through the use of "Logformat: rrdtool" the generation of the .png files is omitted, the .png files will only be generated on-demand. This is done with the help of the CGI-script mrtg-rrd.cgi, which is already installed in /usr/lib/cgi-bin and ready to be used.

As it is a perlscript you can check for the line

my $conffile = "/etc/mrtg-rrd.conf";

somewhere near the begin of it.

Permission/ownership should be

-rwxr-xr-x 1 www-data www-data 28160 2009-08-13 09:57 /usr/lib/cgi-bin/mrtg-rrd.cgi

More information can be found on http://www.fi.muni.cz/~kas/mrtg-rrd/.

Creating entries for systems which should be observed with MRTG is done with the help of cfgmaker (details are in its manpage). The syntax of the configuration could be checked with mrtg -check

MRTG is scheduled with cron, so no service/daemon can be started, stopped, restarted etc. In this case there is an entry in /etc/cron.d/mrtg.

Logging goes to /var/log/mrtg/.

 

5. SSMTP

SSMTP is an extremely simple, lightweigt, no-daemon send-only MTA to a smarthost (which should be enough for this kind of setup).

Configuration is done in /etc/ssmtp/ssmtp.conf

See more details on its manpage.

 

6. RRDTool

Round-Robin database for Smokeping, MRTG, Weathermap4rrd und Nagios. No configuration necessary. Documentation on http://oss.oetiker.ch/rrdtool/.

 

7. Nagios

Nagios is a powerful, highly configurable Monitoring- and Alarmingsystem, which can monitor a wide variety of systems (network, server, daemons, applications). Monitoring could be done for instance for availability or utilization. The monitoring could be restricted to services which are connectable from the outside (e.g. a webserver on port 80/tcp), or with the help of NRPE (Nagios Remote Plugin Executor) plugins for testing could also be executed remote.

There is a lot of info about nagios in the net, so feel free to use your preferred search engine to find whatever you need about it.

 

Lighttpd

Nagios uses a webserver, to communicate with its users, again in our case Lighttpd. There have to be added some more details to Lighty to run with Nagios:

To /etc/lighttpd/lighttpd.conf append:

alias.url +=     ( "/nagios2/cgi-bin" => "/usr/lib/cgi-bin/nagios2" ) 
alias.url +=     ( "/nagios2/stylesheets" => "/etc/nagios2/stylesheets" )
alias.url +=     ( "/nagios2" => "/usr/share/nagios2/htdocs" ) 

$HTTP["url"] =~ "^/nagios2/cgi-bin" { 
        cgi.assign = ( "" => "" ) 
} 
$HTTP["url"] =~ "nagios2" { 
        auth.backend = "htpasswd" 
        auth.backend.htpasswd.userfile = "/etc/nagios2/htpasswd.users" 
        auth.require = ( "" => ( 
                "method" => "basic", 
                "realm" => "Nagios Access", 
                "require" => "user=nagiosadmin|user=helpdesk" 
                ) 
        ) 
        setenv.add-environment = ( "REMOTE_USER" => "www-data" ) 
} 

These entries bridge the gap between Nagios (which is most often running under Apache) and Lighttpd, and implement 2 accounts (nagiosadmin, helpdesk) for logging into Nagios.

The passwords for these accounts are created with htpasswd from the package apache2-utils:

htpasswd /etc/nagios2/htpasswd.users nagiosadmin

 

Nagios

Configuration of Nagios itself is done under /etc/nagios2. The main configfile is nagios.cfg. After modifications in the nagios configuration it would be wise to syntax-check it before you load it (pre-flight check):

In the directory /etc/nagios2 do a

nagios2 -v nagios.cfg

As Nagios also runs as a service/daemon (like Lighthttpd and Smokeping), it has to be reloaded after modifications in its configuration. This is done by

/etc/init.d/nagios2 reload

 

8. Weathermap4rrd

Weathermap4rrd is an application which creates a map of the networkweather, which is another form of visualization of the same data MRTG also visualizes. In a weathermap commonly there is a visualization of the most vital networknodes (routers) and the transmission paths between the nodes. The paths between the nodes are coloured in a way to show the load they are transporting, the nodes could also be colored in a way to show their health (maybe green for ok, red for down). So its possible to get a quick but nevertheless representive overview in one picture over a complex structured net and the datastreams in it. In contrast to MRTG only the newest data is visualized, there is no history.

Weathermap4rrd simply reads the data MRTG collects for the configured routers in the dedicated .rrd files. As MRTG polls its destination in 5 minute intervals, actualization of the weathermap also makes sense in 5 minute intervals. MRTG is scheduled with cron, in the easiest case this is also done with weathermap4rrd.

The cron entry for mrtg is in /etc/cron.d/mrtg and looks like

*/5 *   * * *   root    if [ -d /var/lock/mrtg ]; then if [ -x /usr/bin/mrtg ] && [ -r /etc/mrtg.cfg ]; then env LANG=C /usr/bin/mrtg /etc/mrtg.cfg >> /var/log/mrtg/mrtg.log 2>&1; fi else mkdir /var/lock/mrtg; fi

By appending a line like

*/5 *   * * *   root if [ -x /usr/bin/weathermap4rrd ]; then /usr/bin/weathermap4rrd 2>&1; fi

/usr/bin/weathermap4rrd is the binary which produces the weathermap according to its configfile in /etc/weathermap4rrd/weathermap.conf. In the configfile all configuration for weathermap4rrd is done, what kind of grapicfile is produced, where in the filesystem it is stored, which nodes and communication paths should be on the map, where the .rrd files are located whih should be evaluated, and so on. For details please have a look into the documentation of it on http://weathermap4rrd.tropicalex.net/documentation.php.

 

Weathermap4rrd PHP Version

When you decide to use the php version of weathermap4rrd (not in the Ubuntu repos), you first have to install some packages:

For Lighttpd: apt-get install php5-cgi php

For Weathermap4rrd-php: apt-get install php5-gd

In /etc/lighttpd/lighttpd.conf add:

   "mod_rewrite",
        "mod_redirect",
        "mod_fastcgi",
        "mod_access",

to the server.modules section of the configfile.

In /etc/lighttpd/conf-enabled a new symlink has to be created:

lrwxrwxrwx 1 root root 40 2009-08-13 11:06 10-cgi.conf -> /etc/lighttpd/conf-available/10-cgi.conf

and Lighty eventually has to be restarted.

The archive of the php-version of weathermap4rrd has to be downloaded from http://weathermap4rrd.tropicalex.net/ and to be unpacked directly under the documentroot of Lighty.

Configuration is in the directory documentroot/weathermap*/weathermap.conf and is syntactically very similar to the configuration of the perlversion (but has some additional capabilities).

The php version runs on demand (when the index.php script is loaded into the browser), the weathermap is automatically refreshed (so we do not need any crontab entries) and has some enhancements compared against the perl version.

9. Links

http://www.ubuntu.com/products/whatisubuntu/serveredition/jeos

https://help.ubuntu.com/community/JeOS

https://help.ubuntu.com/community/JeOS#Installing%20VMware%20Tools)

http://www.nagios.org/

http://oss.oetiker.ch/smokeping/

http://oss.oetiker.ch/mrtg/

http://www.fi.muni.cz/~kas/mrtg-rrd/

http://oss.oetiker.ch/rrdtool/

http://www.lighttpd.net/

http://redmine.lighttpd.net/projects/1/wiki/NagiosRecipe

http://weathermap4rrd.tropicalex.net/

ftp://ftp.debian.org/debian/pool/main/s/ssmtp/


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by Masoud (not registered) on Fri, 2009-10-16 10:16.

Can you please tell me that can I install these software on another version of ubuntu like ubuntu 9.04 desktop edition? Because I tried to install on jeos 8.04, but I couldn't configure then network card. It seem it is not recognizing NIC. My laptop is Acer Travelmate.

 Your kindly response is highly appreciated.

Regards

Masoud

Submitted by gbi (registered user) on Wed, 2009-10-28 13:33.

@Masoud,

 I have not tried to use another platform, but I'm sure that you can install and run these apps on nearly every other available Linux Distri, and also on *BSD or commercial Unices which are supported by the applications (but of course you will have to do some modifications in the setup). It should also be possible to use Apache as webserver, or maybe nginx or any other webserversoftware with an appropriate feature set.

I have used Ubuntu Jeos because I wanted to build an appliance running under VMware, and I used 8.04 because it's the newest LTS version. My tutorial is a kind of cooking recipe, so feel free to modify it according to your needs :-) 

Sorry if this is not the answer you expected :-)

Submitted by gbi (registered user) on Thu, 2009-10-15 10:31.

@NetDiva,

thanks for this hint. I have a look at MonitoringForge, and if there is a chance to publish this tutorial, I'll do ;-)

 

@Bill,

1. feel free to install whichever editor you need ;-)

2.-4. I did not want to make a detailed description, only kind of a cooking recipe, where one who is interested has to work out the details for it's use.

 But nevertheless thanks for your hints.

 

@DUCKFACE,

 I have no usecase for ISPconf 2, but feel free to use my tutorial as base for one of you :-)

 

 @DP + anonymous,

I have no knowhow regarding OSSIM, but according to its specs it plays in another league 

 

This appliance is used in a  environment with more than 200 network devices, 100 servers scattered in a township of  ~200000 people. Of course only the most important, critical devices are monitored.

Submitted by NetDiva (not registered) on Tue, 2009-10-06 18:59.

Hi All,
 You can find more monitoring documentation on http://MonitoringForge.org.  In fact, we've also announced a call for monitoring docs.  http://monitoringforge.org/plugins/mwiki/index.php/Www/WTFMM
Cheers!
 Amy Abascal

Submitted by Bill Huber (not registered) on Sun, 2009-10-04 15:47.

Hey, thanks for the great article. I had been looking at trying out GroundWork Monitor, http://www.groundworkopensource.com/products/community-edition/index.html, again. My major complaint on GroundWork Monitor is that it is overkill for small network monitoring needs so this solution looks very appealing since I do not need all of the bells and whistles of a full blown system. Here are some suggestions I have for improving the article.

  1. One of the first things I needed to install to make this installation go smoother was an editor so I could cut-and-paste to my ssh session. In my case I installed nano.
  2. The first application I configured was smokeping. The configuraton file is pretty easy to figure out and can be found at /etc/smokeping/config.  If everything works you can see your results at http://yourip/cgi-bin/smokeping.cgi.
  3. Configuring Nagios is a bit more complicated. The configuration files are located at /etc/nagios2/conf.d. The web page can be found at http://yourip/nagios2/. The Nagios QuickStart Document, http://nagios.sourceforge.net/docs/3_0/quickstart.html, is good primer for the folks not familiar with Nagios.
  4. The Debian logo did not appear in Nagios next to the localhost. After a little research I figured out that I needed to install nagios-images using apt-get install nagios-images.
Submitted by DUCKFACE (registered user) on Wed, 2009-09-30 13:26.
How abaout make it for ISPConf 2 and / or apache :)
Submitted by dp (not registered) on Tue, 2009-09-29 16:25.

I have not tried your solution out yet. I am thinking of installing on its own box for now (no vmware). I am so sick of OSSIM not working correctly I hope your solution works out!

 

Thanks

Submitted by Anonymous (not registered) on Fri, 2009-10-02 20:46.

I wish I can add these functions to Ebox Platform; instead of wasting another box.

To compare with OSSIM is talking about different animals happened to swim in the same zoo.